Commit Graph

432 Commits (fb63228bd36b8b544d6a987a5e2ec0557f787ba6)

Author SHA1 Message Date
Tõnis Tiigi 9df59934cd
Merge pull request #2218 from tonistiigi/error-suggest
dockerfile: add suggestions to how to fix certain errors
2021-07-07 10:52:07 -07:00
Tõnis Tiigi b055d2d55c
Merge pull request #2057 from ktock/export-compression
exporter: Enable to specify the compression type for all layers of the finally exported image
2021-07-06 21:52:36 -07:00
ktock 3152eab58f Enable to forcefully specify compression type
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2021-07-07 12:00:06 +09:00
Tonis Tiigi 4e4152832c dockerfile: remove unnecessary error wrappings
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-07-02 17:29:32 -07:00
Akihiro Suda 1a7543a105
Merge pull request #2163 from tonistiigi/detect-otel-env 2021-07-03 01:01:34 +09:00
Erik Sipsma 3c4f73768b
Merge pull request #2217 from sipsma/fileop-ref-leak
Fix ref leak if fileop ref fails to mount.
2021-07-01 17:38:55 -07:00
Erik Sipsma f3604724d4 Fix ref leak if fileop ref fails to mount.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-07-01 23:38:13 +00:00
Tõnis Tiigi 7d386663ec
Merge pull request #2216 from sipsma/cleancache
Remove unneeded Finalize method from ImmutableRef.
2021-07-01 16:35:33 -07:00
Erik Sipsma b85ef15689 Remove unneeded Finalize method from ImmutableRef.
Finalize was only used outside the cache package in one place, which
called it with the commit arg set to false. The code path followed
when commit==false turned out to essentially be a no-op because
it set "retain cache" to true if it was already set to true.

It was thus safe to remove the only external call to it and remove it
from the interface. This should be helpful for future efforts to
simplify the equal{Mutable,Immutable} fields in cacheRecord, which exist
due to the "lazy commit" feature that Finalize is tied into.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-07-01 17:54:59 +00:00
Akihiro Suda 8d70a9e098
Merge pull request #2215 from tonistiigi/improve-errors
Improve build errors and remove useless grpc prefix
2021-07-01 14:33:40 +09:00
Tonis Tiigi a27d7c3d81 exec: improve error message on exec errors
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-30 21:10:37 -07:00
Akihiro Suda be8ab28a0a
Merge pull request #2081 from tonistiigi/local-differ-none
Allow none differ on local source to avoid false Dockerfile matches
2021-07-01 09:42:24 +09:00
Cory Bennett b1d441b175 [#2112] progress.FromContext returns a writer factory
this allows progress.Controller to manage the writer lifecycle

Signed-off-by: Cory Bennett <cbennett@netflix.com>
2021-06-28 18:58:29 +00:00
Tonis Tiigi 1d0597e85a handle unconfigured spans without errors
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-22 08:27:35 -07:00
Tonis Tiigi c773a215c1 integration: add common context base to all integration tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-18 20:49:49 -07:00
Sebastiaan van Stijn c325f89fa1
replace uses of deprecated containerd/sys.RunningInUserNS()
This utility was moved to a separate package, which has no dependencies.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-18 11:14:26 +02:00
Tonis Tiigi b10f25944b correctly validate span from context
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-15 21:11:27 -07:00
Tonis Tiigi 8f50bae3f1 replace opentracing with opentelemetry tracers
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-15 21:11:27 -07:00
Tonis Tiigi a7b155e30c llb: don't allow content diff for now
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-11 21:15:10 -07:00
Tonis Tiigi baa4fcdb0f add differ support for local source
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-11 21:15:07 -07:00
Akihiro Suda 817955dc8d
Merge pull request #2116 from tonistiigi/git-subdir
git: support subdir component
2021-06-08 14:27:50 +09:00
Tõnis Tiigi 8df5671844
Merge pull request #2082 from aaronlehmann/copy-include-exclude
Add IncludePatterns and ExcludePatterns options for Copy
2021-06-01 12:47:13 -07:00
Tõnis Tiigi e51dbe9fae
Merge pull request #2123 from hinshun/abs-mountpoint
Transform relative mountpoints for exec mounts in the executor
2021-05-26 19:37:52 -07:00
Aaron Lehmann b27fc6169b Combine include/exclude caps into a single cap
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-05-26 13:48:03 -07:00
Aaron Lehmann 626744c9e7 Cache awareness of IncludePatterns/ExcludePatterns
Consider IncludePatterns and ExcludePattern when calculating content
hashes.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-05-26 13:48:03 -07:00
Aaron Lehmann b545c281c1 Add capabilities for copy include/exclude patterns
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-05-26 13:48:03 -07:00
Aaron Lehmann 7660af2f1d Add IncludePatterns and ExcludePatterns options for Copy
Allow include and exclude patterns to be specified for the "copy" op,
similarly to "local".

Depends on https://github.com/tonistiigi/fsutil/pull/101

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-05-26 13:48:03 -07:00
Edgar Lee 9aeed7b193 Transform relative mountpoints for exec mounts in the executor
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2021-05-20 15:17:27 -07:00
Tonis Tiigi 3bdb6b5159 git: support subdir component
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-05-19 22:13:04 -07:00
Vlad A. Ionescu 60d38f972c Move config under worker config. Add new integration test to GHA matrix.
Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>
2021-05-13 14:55:36 +03:00
Vlad A. Ionescu b3cf7c43cf Switch to Acquire API.
Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>
2021-05-12 13:41:33 +03:00
Vlad A. Ionescu 489e17aea9 Add a configuration item to limit parallelism.
Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>
2021-05-10 15:47:59 +03:00
Tõnis Tiigi f113b49307
Merge pull request #2086 from thaJeztah/add_all_proxy
Add "ALL_PROXY" to list of default args / proxy env-vars
2021-04-26 09:09:44 -07:00
Sebastiaan van Stijn a743d4ba02
Add "ALL_PROXY" to list of default args / proxy env-vars
Relates to a82fff6377/docs/packages.md (proxies)

> (..) the first four of these are the standard built-in build-arg options
> available for `docker build`
> (..) The last, `all_proxy`, is a standard var used for socks proxying. Since
> it is not built into `docker build`, if you want to use it, you will need to
> add the following line to the dockerfile:
>
>     ARG all_proxy

Given the we support all other commonly known proxy env-vars by default, it makes
sense to add this one as well.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-24 18:19:02 +02:00
Aaron Lehmann 61bb15a47f Add hack to preserve Dockerfile RUN cache compatibility after mount cache bugfix
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-04-14 08:53:08 -07:00
Aaron Lehmann 0e106e6967 Fix missing mounts in execOp cache map
A bug in cloneExecOp prevented mounts from being included in the digest
computed for the execOp cache map. This could lead to an exec being
wrongly cached when a different mount was used for a subsequent
execution.

Repro case:
https://gist.github.com/aaronlehmann/cfeaefc028df8127fb85b9b5f9125f2d

In this example, pass2 should generate an empty diff because the /from
and /to mounts are the same busybox image. But before the fix, it uses
the cached result from pass1 (with different mounts) instead.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-04-13 22:04:28 -07:00
Tonis Tiigi 86154daec1 solver: check for input index existence
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-04-07 22:02:42 -07:00
Sebastiaan van Stijn 292685db37
solve: remove duplicate filepath.Join
I saw these, and there was no comment in the code explaining if there was a reason
for this; these were added in 2be999ba52, but couldn't
find a mention about these particular lines in the review comments, and I couldn't
think of a special reason for it ':-)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-29 13:24:55 +02:00
Tonis Tiigi bee77d81b9 solver: check for edge dependency before merging
In some cases edges that depend on each other
can have identical cache keys. This happens for
example when empty layers are optimized out by
the differ between identical commands. We need to
detect this case and avoid merge.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-03-18 19:55:35 -07:00
Akihiro Suda 13ea18ad08
vendor containerd (requred for rootless overlayfs on kernel 5.11)
Required for rootless overlayfs on kernel 5.11
https://github.com/containerd/containerd/pull/5076

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-03-10 13:51:27 +09:00
Akihiro Suda 19d31737fb
Merge pull request #1963 from tonistiigi/release-count-fixes 2021-02-19 08:22:54 +09:00
Tonis Tiigi 8d70777537 fileop: fix checksum to contain indexes of inputs
Cache mismatch can happen if fileop switches input indexes
between different actions.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-03 22:17:57 -08:00
Tonis Tiigi 42fb2a8771 avoid double release of same ref released multiple times
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-03 10:53:19 -08:00
Tonis Tiigi b4b0ece384 make sure result refs returned in errors are clone
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-03 10:49:38 -08:00
Tonis Tiigi 3660e5f9c8 don't commit cache mounts on error
Cache mount instances are shared between multiple vertextes/builds
so if one of the cloned instance gets committed reference count
will get corrupted as other parts of the code still see reference as
mountable.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-02 23:20:44 -08:00
Tonis Tiigi dae8f156da update qemu emulators
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-01-21 18:33:00 -08:00
Tõnis Tiigi dd7fe19951
Merge pull request #1830 from coryb/unknown-exit-status
move UnknownExitStatus to executor package from errdefs
2020-12-28 16:35:14 -08:00
Tonis Tiigi 3889ef509c allow nil return from frontend
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-14 17:19:41 -08:00
Tibor Vass bea286f94c
Merge pull request #1871 from tonistiigi/scheduler-fix-for-testing
solver: fix slow-cache status tracking with preprocess
2020-12-11 11:47:45 -08:00
Tonis Tiigi 2f247f0340 solver: fix slow-cache status tracking with preprocess
Preprocess should not be taken into account when calculating
cache-slow state. But it should be used to detect if dep is complete
so preprocess runs before parent's exec function.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-10 22:57:30 -08:00