Commit Graph

721 Commits (69bc68b46ba0518c3d2eefa966badac967fc3e35)

Author SHA1 Message Date
Petr Štetiar f024b4c83d scripts/ubinize-image.sh: fix buildbot breakage
New Docker based buildslaves install just bare minimum of packages, thus
not having bsdmainutils package installed which provides `hexdump`
utility, leading to the following build breakage on buildbots:

 ubinize-image.sh: 12: /builder/scripts/ubinize-image.sh: hexdump: not found

So this patch simply replaces `hexdump` with `od` utility provided by
coreutils package, which should be likely available.

Co-authored-by: Yousong Zhou <yszhou4tech@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
(cherry picked from commit c6d41c320c)
2019-10-31 09:00:13 +01:00
Jo-Philipp Wich a68be42f99 scripts: ipkg-make-index.sh: dereference symbolic links
Use `stat -L` instead of `ls -l` to follow symbolic links when obtaining
the file size of .ipk archives.

Without this change, the size of the symlink, not the size of the target
file is encoded in the package index file.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
(cherry picked from commit ece5cab743)
Fixes: e6af9c017b ("opkg: bump to version 2019-06-14")
[ rmilecki: this has to be backported due to the recent opkg update and
  cb6640381808 ("libopkg: check for file size mismatches") to fix false
  "opkg_install_pkg: Package size mismatch" errors ]
Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
2019-07-22 09:46:49 +02:00
Jeffery To b98b55a64f build: fix STAGING_DIR cleaning for packages
This fixes two issues with cleaning package files from STAGING_DIR:

* CleanStaging currently can only remove files and not directories. This
  changes CleanStaging to use clean-package.sh, which does remove
  directories.

* Because of the way directories are ordered in the staging files list,
  clean-package.sh currently tries (and fails) to remove parent
  directories before removing subdirectories. This changes
  clean-package.sh to process the staging files list in reverse, so that
  subdirectories are removed first.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit a117093679)
2019-01-30 12:40:16 +01:00
Hauke Mehrtens c756d666dc scripts: update config.guess and config.sub
This updates these two files to commit 2fa97a8a0ed3 ("config.guess
(amd64:CYGWIN*:*:*, x86_64:CYGWIN*:*:*): Set master") which is the
current master of
https://git.savannah.gnu.org/gitweb/?p=config.git;a=summary

This contains updates for multiple architectures and will unbreak the
build on the x32 ABI.

Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
(backported from 3361a8817e)
2018-12-18 17:22:07 +01:00
Yousong Zhou fa3805f2b6 scripts/ipkg-build: quash error messages when conffiles.resolved is empty
When Package/xx/conffiles only contains directories that are empty at
package time, conffiles.resolved will be missing and the following
error messages will appear in the build log.

	/home/yousong/git-repo/openwrt/openwrt/scripts/ipkg-build -c -o 0 -g 0 /home/yousong/git-repo/openwrt/openwrt/build_dir/target-mips_24kc_musl/linux-malta_be/openvswitch-2.10.0/ipkg-mips_24kc/openvswitch-common /home/yousong/git-repo/openwrt/openwrt/bin/packages/mips_24kc/packages
	mv: cannot stat 'CONTROL/conffiles.resolved': No such file or directory
	chmod: cannot access 'CONTROL/conffiles': No such file or directory

It will not break the ipkg-build process.  The change is merely cosmetic
to not cause confusion when reading logs

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
(backported from 9e58c20ec9)
2018-12-18 11:28:12 +01:00
Yousong Zhou f5cf9b265f scripts/metadata.pm: avoid adding dup names in provides list
The need arises from building Open vSwitch kernel datapath modules, e.g.

 - kmod-openvswitch from Linux upstream
 - kmod-openvswitch-intree from openvswitch source code

where both provides virtual package "kmod-openvswitch" for userspace
packages to select and depend on

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
(backported from 204081670b)
2018-12-18 11:28:12 +01:00
Felix Fietkau 42dafa0ac5 build: add support for enabling the rootfs/boot partition size option via target feature
Signed-off-by: Felix Fietkau <nbd@nbd.name>
(backported from 00f030a9c6)
2018-12-18 11:28:12 +01:00
Jo-Philipp Wich 2890abbcc0 scripts: time.pl: avoid hard Time::HiRes dependency
Use Time::HiRes when available and fallback to raw syscall interface
when not. If that fails too, simply report 0, 0 as real time.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
(backported from a692e4e3de)
2018-12-18 11:01:55 +01:00
Jo-Philipp Wich b4454ca4fa build: remove GNU time dependency
Replace the GNU time program invocation with a simple Perl script reporting
the timing values. Since we require Perl anyway for the build system, we can
as well use that instead of requiring a random GNU utility rarely installed
by default.

Fixes: ff6e62b288 ("build: log time taken by each packages/steps")
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
(backported from d3ddf6631e)
2018-12-18 11:01:54 +01:00
Yousong Zhou 24bf0c63dc fixup-makefile.pl: fixup when PKG_SOURCE is defined elsewhere
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
(backported from eb6759cc78)
2018-12-18 09:44:04 +01:00
Ted Hess 2b51cac21a scripts: Replace obsolete POSIX tmpnam in slugimage.pl with File::Temp function
Signed-off-by: Ted Hess <thess@kitschensync.net>
(backported from 7590c3c58f)
2018-12-18 09:43:57 +01:00
Jo-Philipp Wich 55d078b249 script: ipkg-build: honour $SOURCE_DATE_EPOCH
When the SOURCE_DATE_EPOCH environment variable is set, use it to
override the timestamps of .ipk archive contents.

This ensures that .ipk archives built in environments without SCM
metadata (mainly the SDK) are reproducible between different runs.

Ref: https://github.com/openwrt/packages/issues/6954
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
(cherry picked from commit d157a76c67)
2018-11-08 11:55:45 +01:00
Jo-Philipp Wich 3315558870 scripts: bundle-libraries: fix logic flaw
Previous refactoring of the script moved the LDSO detection into a
file-not-exists condition, causing onyl the very first executable to
get bundled.

Solve the problem by unconditionally checking for LDSO again.

Fixes: 9030a78a71 ("scripts: bundle-libraries: prevent loading host locales")
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
(cherry picked from commit 5ebcd32997)
2018-08-30 11:32:16 +02:00
Jo-Philipp Wich 686c6c5ca7 scripts: bundle-libraries: prevent loading host locales (FS#1803)
Binary patch the bundled glibc library to inhibit loading of host locale
archives in order to avoid triggering internal libc assertions when
invoking shipped, bundled executables.

The problem has been solved with upstream Glibc commit
0062ace229 ("Gracefully handle incompatible locale data") but we still
need to deal with older Glibc binaries for some time to come.

Fixes FS#1803
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
(cherry picked from commit 9030a78a71)
2018-08-30 11:32:16 +02:00
Matthias Schiffer 514a4b3e1b
include/feeds.mk: rework generation of opkg distfeeds.conf
Allow enabling/commenting/disabling each feed individually by using a
tristate config symbol.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
(cherry picked from commit 16035a7dd3)
2018-07-12 21:26:42 +02:00
Matthias Schiffer a5b9553cf5
scripts/feeds: add src-dummy method
The src-dummy method does not actually obtain any feed, but it can be used
to insert addtional entries into the opkg distfeeds.conf. This is useful to
make package feeds available to users without requiring the corresponding
source feeds to be available during build.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
(cherry picked from commit 6bdd5d8459)
2018-07-12 21:26:41 +02:00
Felix Fietkau 2a7d8cd165 build: fix target metadata scan dependencies
Move SCAN_DEPS to scan.mk to eliminate redundancy with scripts/feeds
Add image/*.mk to SCAN_DEPS for targets to pick up newly added devices

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2018-07-03 11:20:42 +02:00
Philip Prindeville e495a05069 env: only use color diffs on terminals
When sending script/env diff's output to a file or pipeline, you
don't want escape characters for ANSI color sequences fouling the
byte stream.

Signed-off-by: Philip Prindeville <philipp@redfish-solutions.com>
2018-05-05 09:44:43 +02:00
Rosen Penev 1ce846580a download.pl: Change OpenWrt mirrors to HTTPS.
These have supported HTTPS for quite a while. I have not seen any obvious breakage.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2018-05-01 11:12:15 +02:00
Rosen Penev 9ea90209f9 download.pl: Change SourceForge address to HTTPS.
SourceForge has supported HTTPS for its downloads for a long time now.

I have not been able to see any failures resulting from this change.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2018-05-01 11:12:15 +02:00
Felix Fietkau 41ab276fe4 scripts/feeds: add support for git feeds with submodules
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2018-04-27 15:19:19 +02:00
Jo-Philipp Wich 746c590882 scripts: bundle-libraries: fix build on OS X (FS#1493)
This allegedly fixes compilation of the library bundler preload library on
Apple OS X. The resulting executables have not been runtime tested due to a
lack of suitable test hardware.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-04-26 16:06:55 +02:00
Sven Eckelmann 0b20490207 ipq40xx: add support for OpenMesh A62
* QCA IPQ4019
* 256 MB of RAM
* 32 MB of SPI NOR flash (s25fl256s1)
  - 2x 15 MB available; but one of the 15 MB regions is the recovery image
* 2T2R 2.4 GHz
  - QCA4019 hw1.0 (SoC)
  - requires special BDF in QCA4019/hw1.0/board-2.bin with
    bus=ahb,bmi-chip-id=0,bmi-board-id=20,variant=OM-A62
* 2T2R 5 GHz (channel 36-64)
  - QCA9888 hw2.0 (PCI)
  - requires special BDF in QCA9888/hw2.0/board-2.bin
    bus=pci,bmi-chip-id=0,bmi-board-id=16,variant=OM-A62
* 2T2R 5 GHz (channel 100-165)
  - QCA4019 hw1.0 (SoC)
  - requires special BDF in QCA4019/hw1.0/board-2.bin with
    bus=ahb,bmi-chip-id=0,bmi-board-id=21,variant=OM-A62
* multi-color LED (controlled via red/green/blue GPIOs)
* 1x button (reset; kmod-input-gpio-keys compatible)
* external watchdog
  - triggered GPIO
* 1x USB (xHCI)
* TTL pins are on board (arrow points to VCC, then follows: GND, TX, RX)
* 2x gigabit ethernet
  - phy@mdio3:
    + Label: Ethernet 1
    + gmac0 (ethaddr) in original firmware
    + 802.3at POE+
  - phy@mdio4:
    + Label: Ethernet 2
    + gmac1 (eth1addr) in original firmware
    + 18-24V passive POE (mode B)
* powered only via POE

The tool ap51-flash (https://github.com/ap51-flash/ap51-flash) should be
used to transfer the factory image to the u-boot when the device boots up.

The initramfs image can be started using

  setenv bootargs 'loglevel=8 earlycon=msm_serial_dm,0x78af000 console=ttyMSM0,115200 mtdparts=spi0.0:256k(0:SBL1),128k(0:MIBIB),384k(0:QSEE),64k(0:CDT),64k(0:DDRPARAMS),64k(0:APPSBLENV),512k(0:APPSBL),64k(0:ART),64k(0:custom),64k(0:KEYS),15552k(inactive),15552k(inactive2)'
  tftpboot 0x84000000 openwrt-ipq40xx-openmesh_a62-initramfs-fit-uImage.itb
  set fdt_high 0x85000000
  bootm 0x84000000

Signed-off-by: Sven Eckelmann <sven.eckelmann@openmesh.com>
2018-04-23 22:07:22 +02:00
Valentin Spreckels 7a303fe5d1 scripts: add EVA ramboot script
Use the EVA bootloader to load a small linux system into the ram and boot
it from there:

  ./scripts/flashing/eva_ramboot.py 192.168.178.1 path/to/initramfs-kernel.bin

Signed-off-by: Valentin Spreckels <Valentin.Spreckels@Informatik.Uni-Oldenburg.DE>
Acked-by: John Crispin <john@phrozen.org>
[reworded commit message]
Signed-off-by: Mathias Kresin <dev@kresin.me>
2018-03-18 22:22:38 +01:00
Sven Eckelmann 1b773a46c2 build: Allow to change the FIT config section name
Some devices only boot when a special config is found in the image and
completely ignore the default entry during the selection. These devices can
now use the variable DEVICE_DTS_CONFIG in their device image definition.

Signed-off-by: Sven Eckelmann <sven@narfation.org>
2018-03-17 08:09:04 +01:00
Matthias Schiffer 884c9575e1
scripts/metadata.pm: allow group-only Require-User specs
/lib/functions.sh can deal with Require-User specifications that only
contain a group, but no user. Adjust metadata.pm to allow such
specifications as well.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-02-26 07:19:46 +01:00
Alif M. Ahmad 02a1a8af95 build: cleanup leftover qconf files
``make xconfig`` toplevel target will invoke ``make qconf`` inside
./scripts/config directory, which results a ``qconf`` executable.

This commit removes leftover ``qconf`` executable during ``make
config-clean``.

Signed-off-by: Alif M. Ahmad <alive4ever@live.com>
2018-02-13 11:35:35 +01:00
Jo-Philipp Wich a9a43f3d79 build: bundle-libraries.sh: patch bundled ld.so
Remove references to /etc/, /lib/ and /usr/ from the bundled ld.so
interpreter using simple binary patching.

This is needed to prevent loading host system libraries such as
libnss_compat.so.2 on foreign systems, which may result in ld.so
inconsistency assertions.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-02-02 13:59:34 +01:00
Yousong Zhou aa66aa0c9a scripts/qemustart: more portable array operation
The following commands output 1,2,1,1

    cmd0='a=("${a[@]}" 'a'); echo "${#a}"'
    cmd1='a+=('a'); echo "${#a}"'
    bash -c "$cmd0"; zsh -c "$cmd0"
    bash -c "$cmd1"; zsh -c "$cmd1"

The following outputs 0,1,0,0

    cmd2='f() { echo "$#"; }; f "${a[@]}"'
    cmd3="a=(); $cmd2"
    bash -c "$cmd2"; zsh -c "$cmd2"
    bash -c "$cmd3"; zsh -c "$cmd3"

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2018-01-31 16:54:57 +08:00
Jo-Philipp Wich a405a93fa6 metadata: do not emit broken kconfig dependency statements
Prevent emitting broken dependency statements when resolving references
to non existing packages to an empty provider list.

Fixes: 47d6b05ad3 ("metadata: always resolve dependencies through provides list")
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-01-14 19:00:06 +01:00
Matthias Schiffer 06d51dbb9e
scripts/feeds: fix install of packages with different source/binary names
The logic for choice between source and binary packages was reversed.

Fixes: 52719c2b67 "metadata: scripts/feeds: distinguish between source
and binary packages, resolve virtual dependencies"
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-14 17:07:58 +01:00
Matthias Schiffer de0ccd3532
metadata: improve code readability by using (s)printf instead of string interpolation
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:45 +01:00
Matthias Schiffer 4bb54ed15e
build: remove obsolete "package feature" feature
Package "features" seem to be unused for some time. In any case, custom
Config.in snippets and package PROVIDES are a much more flexible way to
express similar options.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:45 +01:00
Matthias Schiffer 52719c2b67
metadata: scripts/feeds: distinguish between source and binary packages, resolve virtual dependencies
Properly resolve build depends to source packages and runtime depends to
binary packages. Dependencies on virtual packages are resolved to the first
provider now.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:45 +01:00
Matthias Schiffer 47d6b05ad3
metadata: always resolve dependencies through provides list
Instead of adding virtual packages to the normal package list, keep a
separate list for provides, make each package provide itself, and resolve
all dependencies through this list. This allows to use PROVIDES to replace
existing packages.

Fixes FS#837.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer 565de8b8a8
metadata: simplify generation of build depends from runtime depends
Runtime depends cannot have a buildtype suffix, and they never refer to
source package names. In addition, this adds warnings about unsatisfiable
dependencies.

Furthermore, this change fixes the generation of conditional build
dependencies for virtual packages provided by different source packages.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer 5b479f7dcf
metadata: handle target build depends together with host build depends
Target build depends are similar to host build depends in that they refer
to source packages rather than binary packages. Therefore, it makes sense
to handle them together, rather than putting them in a list together with
runtime depends and trying to figure out if the entries refer to source or
to binary packages afterwards.

This does lead to PKG_BUILD_DEPENDS entries referring to binary package
names not working anymore, which requires some fixes in the package
repositories.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer 3abf663c22
build: remove package preconfig feature
This feature has been unused for years, and its scope is too limited to be
actually useful.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer 423ec18116
metadata: remove redundant fields from package hash
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer f0c702f124
metadata: replace %subdir hash with a path field in source packages
Every single reference to subdir was concatenated with the source package
name, so it makes sense to store the concatenated value instead.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer 504cb913eb
metadata: change pkg->{src} field to hold a reference
We often want to access fields of a source packages through pkg->{src}.
Allow accessing them directly instead of resolving the source hash through
srcpackages.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer 47f292ff78
metadata: interate over source packages when generating Makefile
All build dependencies are between source packages. Interating over source
rather than binary packages simplifies parts of the code and prepares
further improvement.

As a side effect, this changes the implicit default variant of a few
packages (the first defined is used now instead of the lexicographically
first).

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:44 +01:00
Matthias Schiffer b2ab9c9d33
metadata: move 'builddepends' from binary to source packages
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:43 +01:00
Matthias Schiffer 39be6e7f19
metadata: move 'buildtypes' from binary to source packages
Build types are a property of source rather than binary packages. This is a
preparation for followup cleanup.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:43 +01:00
Matthias Schiffer 19577582e4
metadata: make srcpackage extensible
Turn the srcpackage values into hashes to allow storing more information
than just binary package names.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:43 +01:00
Matthias Schiffer 1a2b5eb7ac
metadata: remove 'base-files' special case
Nothing explicitly depends on base-files, and even if it would, it would
not cause any problems. Remove the unused special case.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-13 19:54:43 +01:00
Sven Eckelmann b02fa3336b om-fwupgradecfg-gen.sh: add support for the A42
Signed-off-by: Sven Eckelmann <sven.eckelmann@open-mesh.com>
2018-01-13 07:58:49 +01:00
Sven Eckelmann 1a384ac0b5 combined-ext-image.sh: generate image in temp dir
The new build commands operate on the input image and use it again as
output image. This conflicts with the way combined-ext-image.sh was
operating. It required that input and output files are different files and
and that it can write freely to the output file.

This can be avoided when all intermediate build steps by
combined-ext-image.sh are done in a temporary directory. The output file is
then only overwritten in the last step.

Signed-off-by: Sven Eckelmann <sven.eckelmann@open-mesh.com>
2018-01-13 07:58:46 +01:00
Matthias Schiffer 0b28cc56d4
scripts/package-metadata.pl: remove broken detection of self-dependencies
A self-dependency is not an error worth a warning; rather, it is very
common: whenever there are dependencies between different binary packages
originating from the same source package, such dependencies occur. Not
actually generating dependency rules is correct, but already handled a few
lines below.

A typo prevented this redundant rule from working, which is the reason the
warning was not actually printed.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2018-01-04 17:18:16 +01:00
Jo-Philipp Wich f4b9d9d6f1 base-files: allow skipping of hash verification
When calling a download target, hash verification is now completely
skipped if we set PKG_HASH=skip.

This allows to easily bump package version:

$ make package/<mypackage>/download PKG_HASH=skip V=s
$ make package/<mypackage>/check FIXUP=1 V=s

This will download the new version of the package, and then automatically
update PKG_HASH with the hash of the new version.  Of course, it is still
the responsibility of the packager to ensure that the new tarball is
legitimate, because it is downloaded from a possibly untrusted source.

Fixes: b30ba14e ("scripts/download.pl: fail loudly if provided hash is unsupported")
Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Acked-by: Stijn Tintel <stijn@linux-ipv6.be>
Signed-off-by: John Crispin <john@phrozen.org>
2017-12-14 09:29:31 +01:00