Commit Graph

257 Commits (23af9dcb86ab24a66bef2534933eb62add91815f)

Author SHA1 Message Date
Sebastiaan van Stijn 0da7d8fdaa
vendor: github.com/docker/docker v20.10.0-beta1.0.20201030232932-c2cc352355d4
full diff: 4634ce647c...c2cc352355

also adds github.com/cilium/ebpf as a dependency, which I set to the same
version as is set in containerd/cgroups version

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-31 00:34:53 +01:00
ktock 0872ddf6aa Delay decoding stargz config for avoiding extra dependency for config pkg
`cmd/buildkitd/config` pkg is imported and used by other packages including
moby/moby.

Though stargz snapshotter configuration is currently effective only with
buildkitd + OCI worker, `cmd/buildkitd/config` consumer needs to introduce an
indirect dependency to stargz snapshotter's config pkg
(`github.com/containerd/stargz-snapshotter/stargz/config`), which is too much.

This commit solves this by delaying decoding the stargz config until OCI
worker's initialization phase.

Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-10-29 23:44:43 +09:00
ktock e3f6e0d249 Bump stargz-snapshotter and partial registry logic integration
Current stargz snapshotter pkg doesn't contain registry configuration and the
client of that pkg needs to pass it to that pkg. So this commit includes changes
of propagating buildkitd's registry configuration to stargz snapshotter.

But this is a partial integration of registry logic between buildkitd and stargz
snapshotter because buildkitd's session-based authentication logic is still not
applied to stargz snapshotter. This means private images that require
`~/.docker/config.json` creds can't be lazily pulled yet.

Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-10-29 12:37:03 +09:00
Tonis Tiigi 34343949b9 debug: enable gc triggers
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-25 21:49:41 -07:00
Tõnis Tiigi 4177e7ca28
Merge pull request #1660 from tonistiigi/token-fetch
auth: fetch tokens from client side
2020-10-12 16:13:19 -07:00
Tonis Tiigi 1f94445456 auth: fetch tokens from client side
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-09-15 13:40:30 -07:00
ktock cf3d695cc2 Enable to run integration tests with stargz snapshotter
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-09-03 16:52:21 +09:00
ktock fb7fe99311 Enable to use stargz snapshotter without spawning plugin process
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-09-03 14:13:26 +09:00
ktock 76189201a8 Add integration test for containerd and stargz snapshotter
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-08-27 15:50:11 +09:00
ktock c975424deb Enable to use remote snapshots for refs
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-08-24 08:44:00 +09:00
Akihiro Suda 8d5e9f8294
new connection helper: podman-container://<CONTAINER>
Similar to `docker-container://` but for Podman containers.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-08-06 15:38:16 +09:00
Akihiro Suda b1db6633bf
Merge pull request #1613 from tonistiigi/env-parsing-fix
buildctl: improve secret parsing
2020-08-02 22:13:54 +09:00
Paul "TBBle" Hampson 883c726bd5 Compile containerd worker on Windows
It's not fully-functional at this time, but it now compiles, which means
we are getting nearer to the point where we can start enabling tests.

This will ensure that it _stays_ compilable over time.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-31 02:49:39 +10:00
Tonis Tiigi 5571a1e605 buildctl: improve secret parsing
type=env,env=foo would not work before

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-28 20:06:00 -07:00
Tonis Tiigi 5da4a40ae8 lint: enable more linters
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:36 -07:00
Tonis Tiigi 06c4197c05 rename binfmt_misc to archutil
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:36 -07:00
Tonis Tiigi c8190b1518 lint: enable golint
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:24 -07:00
Tonis Tiigi f825fea268 buildkitd: ensure stack traces on logging errors
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-17 19:18:49 -07:00
Paul "TBBle" Hampson 156d66da02 Revert "Pin containerd runtime to v1"
This reverts commit 9290c15ffc.

This was pinned during the upgrade to containerd 1.3 series, which
changed the default runtime on Linux to io.containerd.runc.v2.

No specific rationale was listed for this pinning, and clearly it's the
wrong thing to do in the presence of Windows, which does not have this
runtime.

Instead, we rely on the containerd-internal defaults, which distinguish
the runtimes for Linux and Windows.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-16 15:06:20 +10:00
Tõnis Tiigi d3f295c5a9
Merge pull request #1534 from tonistiigi/secrets-env
secrets: allow providing secrets with env
2020-07-06 17:42:42 -07:00
Erik Sipsma 463ec47ba0 client test: Fix check for whether sandbox has containerd
Before this, the check was always returning that containerd wasn't running and
thus skipping the rest of several test cases.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-06-21 20:42:53 -07:00
Tonis Tiigi 0ab180019d secrets: remove file specifics from variable names
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-06-17 12:33:15 -07:00
Tonis Tiigi 64e64e424d secrets: allow providing secrets with env
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-06-16 17:12:39 -07:00
Sebastiaan van Stijn 932c39505b
use containerd/sys to detect UserNamespaces
The implementation in libcontainer/system is quite complicated,
and we only use it to detect if user-namespaces are enabled.

In addition, the implementation in containerd uses a sync.Once,
so that detection (and reading/parsing `/proc/self/uid_map`) is
only performed once.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-06-15 12:51:20 +02:00
Akihiro Suda eb8c8eeb67 buildkitd: prohibit --oci-worker-rootless for real root
Specifying `--oci-worker-rootless` for real root is meaningless and
should be prohibited

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-05-17 10:16:43 +09:00
Sebastiaan van Stijn 3ddd1fc23c
Fix goimports linting
```
[5/5] RUN --mount=target=/go/src/github.com/moby/buildkit 	gometalinter ...
0.435 util/rootless/specconv/specconv_linux.go:1:⚠️ file is not goimported (goimports)
1.320 cache/manager.go:1:⚠️ file is not goimported (goimports)
1.335 cache/manager_test.go:1:⚠️ file is not goimported (goimports)
1.337 cache/migrate_v2.go:1:⚠️ file is not goimported (goimports)
1.342 cache/refs.go:1:⚠️ file is not goimported (goimports)
1.454 cache/remotecache/registry/registry.go:1:⚠️ file is not goimported (goimports)
2.285 cmd/buildctl/build.go:1:⚠️ file is not goimported (goimports)
3.082 executor/oci/user.go:1:⚠️ file is not goimported (goimports)
4.333 session/content/content_test.go:1:⚠️ file is not goimported (goimports)
4.614 snapshot/containerd/content.go:1:⚠️ file is not goimported (goimports)
4.721 solver/errdefs/vertex.go:1:⚠️ file is not goimported (goimports)
6.066 util/network/cniprovider/cni.go:1:⚠️ file is not goimported (goimports)
ERROR: executor failed running [/bin/sh -c gometalinter --config=gometalinter.json ./...]: buildkit-runc did not terminate successfully
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-13 17:38:56 +02:00
Tonis Tiigi 365a58177e buildctl: cleaner typed error printing
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-27 21:25:47 -07:00
Tonis Tiigi 90288ab716 errdefs: update to new packages
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-24 11:25:44 -07:00
Tonis Tiigi 725f5e1207 grpc interceptors for errors
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-22 18:31:32 -07:00
Tonis Tiigi abbda4e941 errdefs: attach source to an error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-21 23:46:17 -07:00
Tonis Tiigi cce301badd solver: attach causing vertex in an error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-21 23:46:17 -07:00
Tonis Tiigi ae3b75d56d errdefs: report component version in stack
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-21 23:46:17 -07:00
Tonis Tiigi 02fff48cbd errdefs: add support for typed errors
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-21 22:57:23 -07:00
Tonis Tiigi e2835e55ad dockerfile: store error location in instructions parser
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-19 20:56:31 -07:00
Tonis Tiigi 37b8832d00 upgrade errors checks to Is()
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-18 22:53:38 -07:00
Tonis Tiigi 1a9d366b49 llb: asyncronous llb graph generation support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-03 14:55:10 -07:00
Tonis Tiigi 56f76a5621 resolver: support self signed certificates
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-03-16 22:09:27 -07:00
Darren Shepherd 6cc8a72665 fix containerd-cni-config-path typo
Signed-off-by: Jacob Blain Christen <jacob@rancher.com>
2020-03-13 09:39:55 -07:00
Tonis Tiigi 2c3cf11fde resolver: update to new registryhosts based config
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-03-05 19:56:43 -08:00
Akihiro Suda 9f90f5a985 rootless: support fuse-overlayfs
While real overlayfs is available only in Ubuntu and Debian kernels,
fuse-overlayfs is universally available for kernel >= 4.18.

For dockerized deployment, `--device /dev/fuse` needs to be added to
`docker run` flags.

Kubernetes deployment needs a custom device plugin that enables
`/dev/fuse`, e.g. https://github.com/honkiko/k8s-hostdev-plugin

Instead of a device plugin, the device can be also enabled by setting
`securityContext.privileged` to `true`.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 11:30:29 +09:00
Tonis Tiigi d1458a6587 update supported platforms without restart
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-02-26 08:15:42 -08:00
Tonis Tiigi e0e29722e2 file: fix compilation on windows
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-01-30 20:21:40 -08:00
Paul "TBBle" Hampson 2bee17a65a Don't always fail euid check on Windows
The check for running as a non-admin euid() doesn't work on Windows,
always returning -1.

For now, treat -1 as "Probably root", and let the failures happen later.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-01-30 00:34:44 +11:00
Paul "TBBle" Hampson 1036fafffa Support npipe the same way we support Unix sockets
The same function used to support Unix sockets automatically supports
Named Pipes on Windows.

This makes the default configuration option for the daemon address work
correctly on Windows.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-01-30 00:34:43 +11:00
Akihiro Suda 1dfd864d22
Merge pull request #1284 from jeffreyhuang23/issue-1200
Fixed issue #1200 (buildctl: add --tlsdir)
2019-12-13 14:15:29 +09:00
Tõnis Tiigi e7304e9a51
Merge pull request #1283 from jeffreyhuang23/issue-1230
Fix issue #1230 (add --oci-worker-binary)
2019-12-12 20:57:59 -08:00
Wei Fu ca1a6c1e6a cmd/buildctl: dump metadata from metadata_v2.db
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-12-13 11:52:01 +08:00
Jeffrey Huang 783a723b22 Fixed issue #1200
Added a command-line flag (tlsdir) to buildctl that allowed for specifying a directory that contains a ca.pem, cert.pem, and key.pem. This command-line flag acts an alias of --tlscacert, --tlscert, and --tlskey. --tlsdir cannot be used at the same time as those flags, and will cause an error if done. The pkg/errors package has been added to cmd/buildctl/common/common.go to allow for the creation of this error message.

Signed-off-by: Jeffrey Huang <jeffreyhuang23@gmail.com>
2019-12-12 15:05:19 -06:00
Nikhil Pandeti cb237bf24e Gofmt and goimports config.go, add documentation
Signed-off-by: Nikhil Pandeti <nikhil.pandeti@utexas.edu>
2019-12-12 12:32:47 -06:00
Jeffrey Huang 20258652b3 Fix issue #1230
To support switching the OCI worker binary via buildkitd and without modifying the Dockerfile, an optional command-line flag was added to main_oci_worker.go that allowed for input the name of an OCI worker binary (ex. crun). This OCI worker binary would then replace the current buildkit-runc with a symbolic link to that binary. The above was done using the os/exec package's Command func.

Signed-off-by: Jeffrey Huang <jeffreyhuang23@gmail.com>
2019-12-12 04:33:24 -06:00