zstd with its default settings (compression level -3) compresses better
than bzip2 -9 (which is the default setting), and is an order of magnitude
faster.
I made the following measurements for the most common compression tools
(all standard Debian Buster versions, default flags unless noted
otherwise), using the debug information of a large x86-64 kernel with
ALL_KMODS:
* kernel-debug.tar: 376M
* kernel-debug.tar.gz: 101M, compressed in ~12s
* kernel-debug.tar.bz2: 91M, compressed in ~15s
* kernel-debug.tar.xz: 57M, compressed in ~101s
* kernel-debug.tar.zst: 86M, compressed in ~1s
With zstd, there is still some room for improvement by increasing the
compression, but the slight increase in compression ratio
(22.83% -> 19.46%) does not justify the significant increase in
compression time (about 5 times on my machine) in my opinion.
Note that multithreaded compression (-T argument) does not affect
reproducibility with zstd.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Don't put CC_HAS_ASM_GOTO, IS_GCC, IS_CLANG and GCC_VERSION runtime
config options into the kernel configs during reconfiguration as it
makes no sense, since these options should be set at runtime.
Fixes: FS#2588
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Commit 965f341aa9 ("build: fix host menu config targets using
ncurses") has moved host's path with pkg-config (usually /usr/bin) at
the first place in PATH variable, which is now causing issues with bison
as BISON_PKGDATADIR points into STAGING_DIR_HOST, but the actual bison
used is the one under host PATH (usually /usr/bin/bison), leading to the
following strange failures:
$ make target/linux/clean kernel_menuconfig V=sc
export MAKEFLAGS= ;make -C /somewhere/linux-4.19.81 menuconfig
make -f ./scripts/Makefile.build obj=scripts/kconfig menuconfig
...
bison -oscripts/kconfig/zconf.tab.c -t -l scripts/kconfig/zconf.y
staging_dir/host/bin/m4: cannot open `staging_dir/host/share/bison/bison.m4': No such file or directory
staging_dir/host/bin/m4: cannot open `staging_dir/host/share/bison/c-skel.m4': No such file or directory
...
gcc -Wp,-MD,scripts/kconfig/.zconf.tab.o.d <...snip...> -o scripts/kconfig/zconf.tab.o scripts/kconfig/zconf.tab.c
gcc: error: scripts/kconfig/zconf.tab.c: No such file or directory
gcc: fatal error: no input files
Fix this by forcing usage of bison under STAGING_DIR_HOST/bin via YACC
make variable.
Cc: Thomas Albers <thomas.gameiro@gmail.com>
Cc: Stijn Tintel <stijn@linux-ipv6.be>
Cc: Eneas U de Queiroz <cotequeiroz@gmail.com>
Tested-by: Ivan Revyakin <LovingFox@GMail.com>
Tested-by: Thomas Albers <thomas.gameiro@googlemail.com>
Ref: https://forum.openwrt.org/t/bpi-r64-kernel-4-19-kernel-menuconfig-error
Fixes: 965f341aa9 ("build: fix host menu config targets using ncurses")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Pass suitable pkg-config overrides to the kernel build process in
order to let our pkg-config wrapper discover libraries provided
by tools/.
This mainly affects the use of libelf which is required for the
CONFIG_STACK_VALIDATION features. So far, the build system either
silently used host system libraries or kbuild simply disabled the
feature due to the lack of a suitable libelf.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
An incorrect variable name was referenced in KERNEL_FILE_DEPENDS, leading
to the omission of the backport-* patch dirs in the generation of the
prepared stamp name.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
The Download/git rule will do a `git checkout <git-ref>`.
So, we can use any ref we want.
No need to limit just to branches.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Commit 86c966a8ae caused HOST_LOADLIBES to
include -lncurses. This was added for fixing build issues on macOS.
This introduces issues on Linux when wide-character ncurses is being
used for compiling, but the non-wide-character version is linked in.
Fix this by adding the extra override for HOST_LOADLIBES only on macOS.
Signed-off-by: Felix Fietkau <nbd@nbd.name>
This allows packages to use kernel make options without the forced
-C $(LINUX_DIR). It also makes it more clear that it to be called from
kernel module packages directly.
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Before SUBDIR was set to $(PATCHVER) which may
or may not include the minor version number of
the linux kernel version. Usually it doesn't.
So the git-clone'd linux kernel was packed without
the minor version number taken into account, which
broke further processing, as it expected the
extracted dir being named linux-$(LINUX_VERSION)
(=with minor version) rather than linux-$(PATCHVER)
(=without minor version).
Changing SUBDIR to $(LINUX_VERSION) creates
consistent behaviour here.
Signed-off-by: Mirko Vogt <mirko-openwrt@nanl.de>
Utilize the existing git download logic from include/download.mk and migrate
the kernel download over to it. This avoids repeatedly cloning kernel sources
after a make target/linux/clean for instance.
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name> [fix build error]
The build system overrides HOST_LOADLIBES to add the staging dir to the
library search path. menuconfig needs -lncurses, add another override
for it.
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Pass KERNEL_FILE_DEPENDS to rdep instead of PKG_FILE_DEPENDS, which is
empty. Also don't pass $(CURDIR) as the directory to timestamp, as it
would also pick up non kernel related changes like image building code.
Should fix kernel being rebuild for unrelated changes, as well as not
being rebuild for changes in target/linux/generic.
Fixes: 22ef1c83b3 ("kernel: make the kernel build auto-clean the build dir like package build")
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
The --mtime argument to 'tar' sets the modification time for all files within
the archive, which determines the timestamp files will get when they are
extracted. In this case, rootfs and other tarballs will get mtimes which
correspond to the last commit timestamp of the build system, as reported by
git/subversion.
This is a step towards reproducible image builds.
Signed-off-by: bryan newbold <bnewbold@robocracy.org>
Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
Signed-off-by: Felix Fietkau <nbd@openwrt.org>
SVN-Revision: 48586
Recent kernels started to mark exported symbols as global.
Adapt expressions in kernel-build.mk to also match global symbols
when grep'ing through nm output.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
SVN-Revision: 42555
When using the options EXTERNAL_KERNEL_TREE or KERNEL_GIT_CLONE_URI,
the command "make downloads" fails as it tries to download the kernel
tarball despite the option. This doesn't happen during a regular build
as in that case, the dependency is conditionned through the LINUX_SITE
variable, which is not set in these cases.
Below is a snapshot of the error for an target using a 3.14 kernel:
make[3]: *** No rule to make target `.../dl/linux-3.14.tar.xz',
needed by `download'. Stop.
Change-Id: I1244969c1bbf9c81a6a64d68ae88ac58b0f8e79e
Signed-off-by: Mathieu Olivari <mathieu@qca.qualcomm.com>
SVN-Revision: 40913
GNU grep has a high per-expression setup overhead when compiling regular
expressions. Use -F to force it to interpret the input as fixed strings,
which is much faster (fraction of a second instead of multiple minutes).
Signed-off-by: Felix Fietkau <nbd@openwrt.org>
SVN-Revision: 39049
If initramfs is enabled, build a second pass kernel containing the CPIO
rootfs, preliminary work to get non exclusive enabling of initramfs.
Signed-off-by: Florian Fainelli <florian@openwrt.org>
SVN-Revision: 37047
Sergio E. Nemirowski
Matthias Schiffer
Felix Fietkau
Petr Štetiar
Jo-Philipp Wich
Alexandru Ardelean
John Crispin
Mirko Vogt
Florian Fainelli
Jonas Gorski
Felix Fietkau
John Crispin
Florian Fainelli
Jo-Philipp Wich
Lars-Peter Clausen
Gabor Juhos