13ea18ad08
vendor containerd (requred for rootless overlayfs on kernel 5.11)
...
Required for rootless overlayfs on kernel 5.11
https://github.com/containerd/containerd/pull/5076
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-03-10 13:51:27 +09:00
19d31737fb
Merge pull request #1963 from tonistiigi/release-count-fixes
2021-02-19 08:22:54 +09:00
8d70777537
fileop: fix checksum to contain indexes of inputs
...
Cache mismatch can happen if fileop switches input indexes
between different actions.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-03 22:17:57 -08:00
42fb2a8771
avoid double release of same ref released multiple times
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-03 10:53:19 -08:00
b4b0ece384
make sure result refs returned in errors are clone
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-03 10:49:38 -08:00
3660e5f9c8
don't commit cache mounts on error
...
Cache mount instances are shared between multiple vertextes/builds
so if one of the cloned instance gets committed reference count
will get corrupted as other parts of the code still see reference as
mountable.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-02 23:20:44 -08:00
dae8f156da
update qemu emulators
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-01-21 18:33:00 -08:00
dd7fe19951
Merge pull request #1830 from coryb/unknown-exit-status
...
move UnknownExitStatus to executor package from errdefs
2020-12-28 16:35:14 -08:00
3889ef509c
allow nil return from frontend
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-14 17:19:41 -08:00
bea286f94c
Merge pull request #1871 from tonistiigi/scheduler-fix-for-testing
...
solver: fix slow-cache status tracking with preprocess
2020-12-11 11:47:45 -08:00
2f247f0340
solver: fix slow-cache status tracking with preprocess
...
Preprocess should not be taken into account when calculating
cache-slow state. But it should be used to detect if dep is complete
so preprocess runs before parent's exec function.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-10 22:57:30 -08:00
1b271e0014
solver: delay before discarding job
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-09 20:53:03 -08:00
9d3f55c400
move ExitError from solver/errdefs to frontend/gateway/errdefs
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-12-07 01:09:59 +00:00
d5f179bb79
Merge pull request #1863 from tonistiigi/solver-slowcache
...
solver: improve slow cache detection logging
2020-12-02 16:47:35 -08:00
706129170d
solver: remove scheduler fix
...
Asked in review.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-02 16:07:13 -08:00
f38972a8ec
mounts: handle optional ssh mount
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-02 14:28:51 -08:00
6a321dc574
solver: improve slow cache detection and add logging
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-01 22:38:15 -08:00
e46cfab9fb
fix invalid cancellation errors on pull
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-01 00:27:31 -08:00
df3a9cad23
make sure execerror is released on cancellation
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-25 15:50:10 -08:00
372df78cc8
Guarantee err results are released when result proxy is released
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-25 11:26:53 -08:00
67bcfe699a
copy containerd.UnknownExitStatus to local const
...
Copy this const to a local constant to prevent importing the containerd
client in the front-end.
For consistency, I also updated the executor code to use the same const,
although not strictly needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-18 00:11:09 +01:00
5e5f527e06
Merge pull request #1732 from hinshun/exec-error
...
Allow gateway exec-ing into a failed solve with an exec op
2020-11-16 19:13:11 -08:00
fa8a02c861
Fix optional cast for WithOp when unit testing
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-16 13:37:30 -08:00
1240dd7795
Return committed readonly inputs and actives in exec error in MountIDs
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-16 13:21:23 -08:00
8dc3a54a78
Merge pull request #1782 from alexcb/acb-other-git-user
...
SSH-based auth for llb.Git operations
2020-11-16 10:53:44 -08:00
3ba6cd7bc2
Fix ExecError.EachRef invoking callback with possibly nil solver.Results
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 23:20:41 -08:00
c33bcd61a2
Rename OutputIDs to MountIDs
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
bcff7baf60
Allow scratch mounts with gateway.RunContainer
...
- Plumb default worker by adding GetDefault() to frontend.WorkerInfos
- To avoid cyclic dependency, refactor frontend.WorkerInfos to worker.Infos
- Refactor gateway.NewContainer to share code with llbsolver/ops/exec.go
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
a459eb4927
Return exec error for errors returned before executor
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
7e1dc9bec1
Refactor to file action indexed outputs
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
2d23d0cc43
Fix lint and unit tests for fileopsolver
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
7ce58c31ba
Plumb op metadata to recreate failed ops with gateway exec
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
fad612e748
Allow worker to override GetRemote(), needed in moby integration
...
This fixes panics in the moby integration of buildkit when using cache.
Panics come from nil Differ in computeBlobChain which is called in GetRemote().
GetRemote() got moved from Worker to ImmutableRef during the lazy-pull refactor.
However, the ability to easily override GetRemote() got lost with that refactor.
This patch attempts to allow for the override while keeping changes minimal.
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-11-13 21:30:35 +00:00
f2c8eb177c
Support cloning from non-standard git repos
...
- fixes assumption that ssh git clones must be via the `git` user.
- allows passing the SSH_AUTH_SOCK from the client to GitSource
- allows passing a known_host entry for ssh
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
2020-11-10 15:55:17 -08:00
e86356ddf8
solver: add input preprocess capability
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-03 14:49:44 -08:00
bdcee17437
executor: change mount to struct
...
Allows readonly passed cleanly.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:51 -08:00
e3b05289d8
add session injection to remote loading
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:44 -08:00
fbe1346912
replace github.com/docker/docker/pkg/locker with moby/locker v1.0.1
...
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 09:25:05 +01:00
1b1d9e88dc
Merge pull request #1747 from tonistiigi/default-shell-path
...
dockerfile: set correct default path and shell based on OS
2020-10-26 16:37:12 +09:00
cac0b68f9e
solver: make sure job id cleared after build
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-25 21:50:40 -07:00
ecf070a027
exec: use platform specific default path
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-23 15:30:23 -07:00
cd42b669c9
Enable parallelization of NewContentHashFunc
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-10-21 11:17:34 -07:00
5e7ae230b2
Add hostname specifying for building
...
Fix : #1301
Signed-off-by: Lu Jingxiao <lujingxiao@huawei.com>
2020-10-14 14:45:36 +08:00
b4fad847ac
frontend: support for subrequests
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-07 13:36:51 -07:00
44f27708b3
frontend: make sure inputs support for frontends is detectable from client
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-04 10:44:16 -07:00
095a919c95
remove unnecessary MountManager interface
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-02 06:24:34 +00:00
03e1c199a3
remove SessonManager from FrontendLLBBridge, adding as arg to Frontend.Solve
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-02 01:59:05 +00:00
3cc8aa0649
move ExecOps mounting logic into new package so logic can be reused from gateway exec
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-09-15 07:01:42 +00:00
355e937e15
update gateway to add ability to run and exec into containers
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-08-31 21:13:59 +00:00
545532ab0e
Merge pull request #1475 from sipsma/lazy-ref
...
Add support for lazily-pulled blobs in cache manager.
2020-08-06 12:54:45 -07:00
862387445e
grpcerrors: use full typeurl registration
...
Also switches current types to json.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-06 12:28:48 -07:00
55cbd19dec
Add support for lazily-pulled blobs in cache manager.
...
This allows the layers of images to only be pulled if/once they are actually
required.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-08-05 17:18:43 -07:00
f4393eaa31
Ensure that time has passed between foo0 and foo1
...
On Windows, the resolution of time.Now() can be surprisingly low: 15ms
according to the comments in the source of time.Now().
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-30 03:21:30 +10:00
dc95525191
vendor: update fsutil to c3ed55f3b4
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-26 23:25:00 -07:00
3c96d0b863
Support not chowning files on all platforms
...
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-23 03:18:30 +10:00
5da4a40ae8
lint: enable more linters
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:36 -07:00
06c4197c05
rename binfmt_misc to archutil
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:36 -07:00
c8190b1518
lint: enable golint
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:24 -07:00
57a88b0afb
lint: add misspell and unused
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-18 08:13:57 -07:00
6a56695d3a
remove `Get` prefix from Executor/CacheManager funcs and update to avoid naming conflicts
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-17 00:54:15 +00:00
453c76d29e
add Run/Exec to FrontendLLBBridge interface
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-16 23:20:14 +00:00
5e91dff4ed
fix error handling for exec when container fails to start
...
update run/exec tests for stdin and expected failures
move common tests for runc and container to shared tests package
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-10 22:06:42 +00:00
6d58121c11
Update Executor interface for Run and Exec
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-09 23:40:36 +00:00
488130002a
Merge pull request #1551 from tonistiigi/session-group
...
session: track sessions with a group construct
2020-07-08 16:37:07 -07:00
dd304ede33
solver: fix marking already cached vertex as cancelled
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-05 12:03:59 -07:00
2e9987ad16
session: track sessions with a group construct
...
Avoid hidden session passing and allow one session to drop when
multiple builds share a vertex.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-05 00:15:25 -07:00
49de675234
git: support for token authentication
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-06-16 17:00:06 -07:00
932c39505b
use containerd/sys to detect UserNamespaces
...
The implementation in libcontainer/system is quite complicated,
and we only use it to detect if user-namespaces are enabled.
In addition, the implementation in containerd uses a sync.Once,
so that detection (and reading/parsing `/proc/self/uid_map`) is
only performed once.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-06-15 12:51:20 +02:00
b7fd382230
add chmod in COPY and ADD command
...
Signed-off-by: Chenbin <chen.bin11@zte.com.cn>
2020-06-10 21:37:46 +08:00
f9e26128c0
exec: use qemu emulator automatically
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-06-07 22:08:52 -07:00
d6f5e972de
Merge pull request #1494 from tonistiigi/errdefs2
...
llb: add source tracking support
2020-05-25 23:15:20 +09:00
7a90a36b46
Support multiple source maps and locations for source-level stacktraces
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-05-20 16:17:35 -07:00
d173e3dca8
pb: add more comments
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-19 17:23:03 -07:00
dd765674fb
solver: gracefully handle cache loading errors
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-18 18:21:34 -07:00
75d64ffb4a
fix proto indentions
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-18 15:50:08 -07:00
6073e6cff3
llb: enable source tracking
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-18 15:50:03 -07:00
1f9599aba3
llb: move source mapping to llb metadata
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-18 15:49:41 -07:00
3ddd1fc23c
Fix goimports linting
...
```
[5/5] RUN --mount=target=/go/src/github.com/moby/buildkit gometalinter ...
0.435 util/rootless/specconv/specconv_linux.go:1:⚠️ file is not goimported (goimports)
1.320 cache/manager.go:1:⚠️ file is not goimported (goimports)
1.335 cache/manager_test.go:1:⚠️ file is not goimported (goimports)
1.337 cache/migrate_v2.go:1:⚠️ file is not goimported (goimports)
1.342 cache/refs.go:1:⚠️ file is not goimported (goimports)
1.454 cache/remotecache/registry/registry.go:1:⚠️ file is not goimported (goimports)
2.285 cmd/buildctl/build.go:1:⚠️ file is not goimported (goimports)
3.082 executor/oci/user.go:1:⚠️ file is not goimported (goimports)
4.333 session/content/content_test.go:1:⚠️ file is not goimported (goimports)
4.614 snapshot/containerd/content.go:1:⚠️ file is not goimported (goimports)
4.721 solver/errdefs/vertex.go:1:⚠️ file is not goimported (goimports)
6.066 util/network/cniprovider/cni.go:1:⚠️ file is not goimported (goimports)
ERROR: executor failed running [/bin/sh -c gometalinter --config=gometalinter.json ./...]: buildkit-runc did not terminate successfully
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-13 17:38:56 +02:00
625f6ecc82
file: fix double-remap of old uid value
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-08 11:33:01 -07:00
da4e440dbd
buidlctl: upon error add line number in addition to filename
...
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-04-29 20:41:26 +00:00
365a58177e
buildctl: cleaner typed error printing
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-27 21:25:47 -07:00
c78e875977
remove last instances of errors cause
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-24 11:25:53 -07:00
90288ab716
errdefs: update to new packages
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-24 11:25:44 -07:00
4b2636acca
dockerfile: add more source information to errors
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-21 23:46:17 -07:00
abbda4e941
errdefs: attach source to an error
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-21 23:46:17 -07:00
cce301badd
solver: attach causing vertex in an error
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-21 23:46:17 -07:00
ae3b75d56d
errdefs: report component version in stack
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-21 23:46:17 -07:00
e4cc0866f5
gateway: fix error details passing through gateway
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-21 23:46:17 -07:00
3f77f0495b
grpc: add error handling wrappers to client/server
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-21 23:46:11 -07:00
02fff48cbd
errdefs: add support for typed errors
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-21 22:57:23 -07:00
37b8832d00
upgrade errors checks to Is()
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-18 22:53:38 -07:00
cfabf3009b
solver: avoid looping over same keys in loadwithparents
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-03-18 17:07:57 -07:00
0010996598
solver: avoid recursive loop on cache-export
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-03-02 22:35:59 -08:00
7b579cdb98
vendor: update containerd (and various packages)
...
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-02-28 10:24:10 +09:00
d1458a6587
update supported platforms without restart
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-02-26 08:15:42 -08:00
7e0f923181
Implement CapFrontendInput to pass llb.States to frontends
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-02-24 14:38:01 -08:00
332cdb5e25
Merge pull request #1355 from tonistiigi/mounts-deadlock
...
ops: fix deadlock on releasing shared mounts
2020-02-21 10:34:57 -08:00
2f5ad30ae5
Merge pull request #1351 from tonistiigi/security-devices
...
mount whitelist of devices on insecure security mode
2020-02-14 16:52:26 +09:00
b0e76973ee
solver: use correct context for getting cache managers lazily
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-02-06 15:11:49 -08:00
8cfe2de889
solver: evaluate solve results lazily
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-02-05 15:29:43 -08:00
Akihiro Suda
Tonis Tiigi
Tibor Vass
Cory Bennett
Edgar Lee
Sebastiaan van Stijn
Tibor Vass
Alex Couture-Beil
Lu Jingxiao
Erik Sipsma
Paul "TBBle" Hampson
Chenbin