334c62f3b7
Transform relative mountpoints for exec mounts in the executor
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
(cherry picked from commit 9aeed7b193
2021-05-26 19:45:29 -07:00
361c2b6aa0
Add hack to preserve Dockerfile RUN cache compatibility after mount cache bugfix
...
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
(cherry picked from commit 61bb15a47f
2021-04-21 23:28:07 -07:00
f1ae3a25b9
Fix missing mounts in execOp cache map
...
A bug in cloneExecOp prevented mounts from being included in the digest
computed for the execOp cache map. This could lead to an exec being
wrongly cached when a different mount was used for a subsequent
execution.
Repro case:
https://gist.github.com/aaronlehmann/cfeaefc028df8127fb85b9b5f9125f2d
In this example, pass2 should generate an empty diff because the /from
and /to mounts are the same busybox image. But before the fix, it uses
the cached result from pass1 (with different mounts) instead.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
(cherry picked from commit 0e106e6967
2021-04-21 23:27:26 -07:00
de7435ac1a
avoid double release of same ref released multiple times
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 42fb2a8771
2021-02-21 20:25:08 +01:00
7c7c7e4f6e
make sure result refs returned in errors are clone
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit b4b0ece384
2021-02-21 20:25:07 +01:00
ce6dc4bd0d
don't commit cache mounts on error
...
Cache mount instances are shared between multiple vertextes/builds
so if one of the cloned instance gets committed reference count
will get corrupted as other parts of the code still see reference as
mountable.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 3660e5f9c8
2021-02-21 20:25:04 +01:00
2f4d8ba7a0
Merge pull request #1968 from thaJeztah/0.8_backport_binfmt_update
...
[v0.8 backport] update qemu emulators
2021-02-10 08:44:52 -08:00
7a4ca33994
fileop: fix checksum to contain indexes of inputs
...
Cache mismatch can happen if fileop switches input indexes
between different actions.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 8d70777537
2021-02-04 22:20:54 +01:00
c3789b226b
update qemu emulators
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit dae8f156da
2021-02-04 22:17:19 +01:00
3889ef509c
allow nil return from frontend
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-14 17:19:41 -08:00
f38972a8ec
mounts: handle optional ssh mount
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-02 14:28:51 -08:00
df3a9cad23
make sure execerror is released on cancellation
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-25 15:50:10 -08:00
372df78cc8
Guarantee err results are released when result proxy is released
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-25 11:26:53 -08:00
1240dd7795
Return committed readonly inputs and actives in exec error in MountIDs
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-16 13:21:23 -08:00
3ba6cd7bc2
Fix ExecError.EachRef invoking callback with possibly nil solver.Results
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 23:20:41 -08:00
c33bcd61a2
Rename OutputIDs to MountIDs
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
bcff7baf60
Allow scratch mounts with gateway.RunContainer
...
- Plumb default worker by adding GetDefault() to frontend.WorkerInfos
- To avoid cyclic dependency, refactor frontend.WorkerInfos to worker.Infos
- Refactor gateway.NewContainer to share code with llbsolver/ops/exec.go
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
a459eb4927
Return exec error for errors returned before executor
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
7e1dc9bec1
Refactor to file action indexed outputs
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
2d23d0cc43
Fix lint and unit tests for fileopsolver
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
7ce58c31ba
Plumb op metadata to recreate failed ops with gateway exec
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
fad612e748
Allow worker to override GetRemote(), needed in moby integration
...
This fixes panics in the moby integration of buildkit when using cache.
Panics come from nil Differ in computeBlobChain which is called in GetRemote().
GetRemote() got moved from Worker to ImmutableRef during the lazy-pull refactor.
However, the ability to easily override GetRemote() got lost with that refactor.
This patch attempts to allow for the override while keeping changes minimal.
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-11-13 21:30:35 +00:00
e86356ddf8
solver: add input preprocess capability
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-03 14:49:44 -08:00
bdcee17437
executor: change mount to struct
...
Allows readonly passed cleanly.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:51 -08:00
e3b05289d8
add session injection to remote loading
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:44 -08:00
fbe1346912
replace github.com/docker/docker/pkg/locker with moby/locker v1.0.1
...
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 09:25:05 +01:00
ecf070a027
exec: use platform specific default path
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-23 15:30:23 -07:00
cd42b669c9
Enable parallelization of NewContentHashFunc
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-10-21 11:17:34 -07:00
5e7ae230b2
Add hostname specifying for building
...
Fix : #1301
Signed-off-by: Lu Jingxiao <lujingxiao@huawei.com>
2020-10-14 14:45:36 +08:00
095a919c95
remove unnecessary MountManager interface
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-02 06:24:34 +00:00
03e1c199a3
remove SessonManager from FrontendLLBBridge, adding as arg to Frontend.Solve
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-02 01:59:05 +00:00
3cc8aa0649
move ExecOps mounting logic into new package so logic can be reused from gateway exec
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-09-15 07:01:42 +00:00
55cbd19dec
Add support for lazily-pulled blobs in cache manager.
...
This allows the layers of images to only be pulled if/once they are actually
required.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-08-05 17:18:43 -07:00
dc95525191
vendor: update fsutil to c3ed55f3b4
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-26 23:25:00 -07:00
3c96d0b863
Support not chowning files on all platforms
...
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-23 03:18:30 +10:00
06c4197c05
rename binfmt_misc to archutil
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:36 -07:00
c8190b1518
lint: enable golint
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:24 -07:00
6a56695d3a
remove `Get` prefix from Executor/CacheManager funcs and update to avoid naming conflicts
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-17 00:54:15 +00:00
453c76d29e
add Run/Exec to FrontendLLBBridge interface
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-16 23:20:14 +00:00
5e91dff4ed
fix error handling for exec when container fails to start
...
update run/exec tests for stdin and expected failures
move common tests for runc and container to shared tests package
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-10 22:06:42 +00:00
6d58121c11
Update Executor interface for Run and Exec
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-09 23:40:36 +00:00
2e9987ad16
session: track sessions with a group construct
...
Avoid hidden session passing and allow one session to drop when
multiple builds share a vertex.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-05 00:15:25 -07:00
932c39505b
use containerd/sys to detect UserNamespaces
...
The implementation in libcontainer/system is quite complicated,
and we only use it to detect if user-namespaces are enabled.
In addition, the implementation in containerd uses a sync.Once,
so that detection (and reading/parsing `/proc/self/uid_map`) is
only performed once.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-06-15 12:51:20 +02:00
b7fd382230
add chmod in COPY and ADD command
...
Signed-off-by: Chenbin <chen.bin11@zte.com.cn>
2020-06-10 21:37:46 +08:00
f9e26128c0
exec: use qemu emulator automatically
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-06-07 22:08:52 -07:00
7a90a36b46
Support multiple source maps and locations for source-level stacktraces
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-05-20 16:17:35 -07:00
6073e6cff3
llb: enable source tracking
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-18 15:50:03 -07:00
625f6ecc82
file: fix double-remap of old uid value
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-08 11:33:01 -07:00
c78e875977
remove last instances of errors cause
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-24 11:25:53 -07:00
37b8832d00
upgrade errors checks to Is()
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-18 22:53:38 -07:00
Edgar Lee
Aaron Lehmann
Tonis Tiigi
Tibor Vass
Sebastiaan van Stijn
Lu Jingxiao
Cory Bennett
Erik Sipsma
Paul "TBBle" Hampson
Chenbin