Commit Graph

45 Commits (ee1ad6893a0a7a2c0a72d82b3ba7a86c46794122)

Author SHA1 Message Date
Sebastiaan van Stijn 0e6251510e
cmd/buildkitd: replace BurntSushi/toml with pelletier/go-toml
The BurntSushi/toml project has been deprecated, and the ecosystem
is converging on using pelletier/go-toml as the "canonical" replacement.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-20 18:57:04 +02:00
Morlay 18b49fd7dc refactor to use util/bklog instead of using logurs directly
Signed-off-by: Morlay <morlay.null@gmail.com>
2021-07-13 11:42:31 +08:00
Tonis Tiigi d512920c24 support collecting traces from llb.Exec
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-20 21:57:35 -07:00
Sebastiaan van Stijn c325f89fa1
replace uses of deprecated containerd/sys.RunningInUserNS()
This utility was moved to a separate package, which has no dependencies.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-18 11:14:26 +02:00
ktock 1764b3b249 Enable stargz snapshotter to use session for snapshot auth
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2021-06-09 16:10:08 +09:00
Vlad A. Ionescu 60d38f972c Move config under worker config. Add new integration test to GHA matrix.
Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>
2021-05-13 14:55:36 +03:00
Vlad A. Ionescu b3cf7c43cf Switch to Acquire API.
Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>
2021-05-12 13:41:33 +03:00
Akihiro Suda ad681c0154
AkihiroSuda/containerd-fuse-overlayfs -> containerd/fuse-overlayfs-snapshotter
The repo has been moved.

Also bumps up containerd library to v1.5.0-beta.4.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-03-23 18:43:01 +09:00
Brian Goff d9834f872c
Add support for apparmor/selinux
Set's an apparmor profile on the OCI spec if one is configred on the
worker.
Adds selinux labels to containers (only added if selinux is enabled on
the system).

This assumes that the specified apparmor profile is already loaded on
the system and does not try to load it or even check if it is loaded.

SELinux support requires the `selinux` build tag to be added.
Likewise, `runc` would require both the `apparmor` and `selinux` build
tags.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>

Vendored go-selinux to v1.8.0
Fixed tests

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 68bb095353)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-04 22:01:34 +01:00
ktock ae1b79bbc6 Bump github.com/containerd/stargz-snapshotter to v0.2.0
This enables BuildKit to lazily pull eStargz with new footer format.

Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-12-10 15:20:57 +09:00
ktock 0872ddf6aa Delay decoding stargz config for avoiding extra dependency for config pkg
`cmd/buildkitd/config` pkg is imported and used by other packages including
moby/moby.

Though stargz snapshotter configuration is currently effective only with
buildkitd + OCI worker, `cmd/buildkitd/config` consumer needs to introduce an
indirect dependency to stargz snapshotter's config pkg
(`github.com/containerd/stargz-snapshotter/stargz/config`), which is too much.

This commit solves this by delaying decoding the stargz config until OCI
worker's initialization phase.

Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-10-29 23:44:43 +09:00
ktock e3f6e0d249 Bump stargz-snapshotter and partial registry logic integration
Current stargz snapshotter pkg doesn't contain registry configuration and the
client of that pkg needs to pass it to that pkg. So this commit includes changes
of propagating buildkitd's registry configuration to stargz snapshotter.

But this is a partial integration of registry logic between buildkitd and stargz
snapshotter because buildkitd's session-based authentication logic is still not
applied to stargz snapshotter. This means private images that require
`~/.docker/config.json` creds can't be lazily pulled yet.

Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-10-29 12:37:03 +09:00
ktock fb7fe99311 Enable to use stargz snapshotter without spawning plugin process
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-09-03 14:13:26 +09:00
ktock c975424deb Enable to use remote snapshots for refs
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-08-24 08:44:00 +09:00
Sebastiaan van Stijn 932c39505b
use containerd/sys to detect UserNamespaces
The implementation in libcontainer/system is quite complicated,
and we only use it to detect if user-namespaces are enabled.

In addition, the implementation in containerd uses a sync.Once,
so that detection (and reading/parsing `/proc/self/uid_map`) is
only performed once.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-06-15 12:51:20 +02:00
Akihiro Suda eb8c8eeb67 buildkitd: prohibit --oci-worker-rootless for real root
Specifying `--oci-worker-rootless` for real root is meaningless and
should be prohibited

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-05-17 10:16:43 +09:00
Tonis Tiigi 2c3cf11fde resolver: update to new registryhosts based config
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-03-05 19:56:43 -08:00
Akihiro Suda 9f90f5a985 rootless: support fuse-overlayfs
While real overlayfs is available only in Ubuntu and Debian kernels,
fuse-overlayfs is universally available for kernel >= 4.18.

For dockerized deployment, `--device /dev/fuse` needs to be added to
`docker run` flags.

Kubernetes deployment needs a custom device plugin that enables
`/dev/fuse`, e.g. https://github.com/honkiko/k8s-hostdev-plugin

Instead of a device plugin, the device can be also enabled by setting
`securityContext.privileged` to `true`.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 11:30:29 +09:00
Jeffrey Huang 20258652b3 Fix issue #1230
To support switching the OCI worker binary via buildkitd and without modifying the Dockerfile, an optional command-line flag was added to main_oci_worker.go that allowed for input the name of an OCI worker binary (ex. crun). This OCI worker binary would then replace the current buildkit-runc with a symbolic link to that binary. The above was done using the os/exec package's Command func.

Signed-off-by: Jeffrey Huang <jeffreyhuang23@gmail.com>
2019-12-12 04:33:24 -06:00
Tonis Tiigi 57d19b1208 network: refactor cni support to multiple packages
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-06 12:52:37 -07:00
Tonis Tiigi e7759a861b client: add cni network tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-10 18:00:06 -07:00
Tonis Tiigi bc7a6f2556 add cni networking support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-10 14:42:30 -07:00
Tibor Vass 8e692507ee executor: allow setting DNSConfig to be used by resolvconf
This patch allows downstream code to pass a DNSConfig that is
then used by executor/oci.GetResolvConf.

This would allow the BuildKit-based builder in Docker to honor
the docker daemon's DNS configuration, thus fixing a feature gap
with the legacy builder.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-17 20:43:33 +00:00
Tonis Tiigi bffb08a1c4 buildkitd: use asyncremove on overlay
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-05-14 14:35:13 -07:00
Tonis Tiigi 6921dbe0f6 snapshot: base for userns remapping
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-02 18:23:46 -07:00
Tonis Tiigi 1dd4a0ace1 buildkitd: error on config load error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-22 11:20:08 -07:00
Akihiro Suda 26010a4864 decouple SessionManager from Worker
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-02-23 22:55:17 +09:00
Tonis Tiigi a1f3f8462c buildkitd: add oci worker gc flags
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-01-22 17:34:40 -08:00
Akihiro Suda c54f4a986d support --oci-worker-no-process-sandbox
Note that this mode allows build executor containers to kill (and potentially ptrace) an arbitrary process in the BuildKit host namespace.
This mode should be enabled only when the BuildKit is running in a container as an unprivileged user.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-01-08 10:42:52 +09:00
Tonis Tiigi ddfb7f77c9 hack: update test dockerfiles to buildkit
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-10 23:26:05 -07:00
Tonis Tiigi ecd4a22546 resolver: add config support for mirrors/plainhttp
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-07 18:44:45 -07:00
Akihiro Suda 756ca6cc6d
Merge pull request #602 from tonistiigi/gc
automatic GC
2018-09-05 10:47:21 +09:00
Tonis Tiigi 079c747c4e ontrol: support calling automatic GC
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-04 18:13:29 -07:00
Ian Campbell 7cb70e1c31 Initialise workers' label maps before assigning.
Otherwise:

    panic: assignment to entry in nil map

    goroutine 1 [running]:
    main.applyOCIFlags(0xc4200e71e0, 0xc420400000, 0x0, 0x0)
    	/go/src/github.com/moby/buildkit/cmd/buildkitd/main_oci_worker.go:97 +0x1ac
    main.ociWorkerInitializer(0xc4200e71e0, 0xc4204104e0, 0xc420400000, 0x43409b, 0x12, 0xc42026b0f8, 0x4337fc, 0xc420000180)
    	/go/src/github.com/moby/buildkit/cmd/buildkitd/main_oci_worker.go:118 +0x50
    main.newWorkerController(0xc4200e71e0, 0xc4204104e0, 0xc420400000, 0xc420422000, 0xe5dc54, 0x11)
    	/go/src/github.com/moby/buildkit/cmd/buildkitd/main.go:520 +0x324
    main.newController(0xc4200e71e0, 0xc420400000, 0x1c0, 0x0, 0x0)
    	/go/src/github.com/moby/buildkit/cmd/buildkitd/main.go:489 +0xdc
    main.main.func3(0xc4200e71e0, 0x0, 0x0)
    	/go/src/github.com/moby/buildkit/cmd/buildkitd/main.go:203 +0x3dd
    github.com/moby/buildkit/vendor/github.com/urfave/cli.HandleAction(0xcdd420, 0xe93e98, 0xc4200e71e0, 0xc4200e71e0, 0xc42026b888)
    	/go/src/github.com/moby/buildkit/vendor/github.com/urfave/cli/app.go:502 +0xc8
    github.com/moby/buildkit/vendor/github.com/urfave/cli.(*App).Run(0xc4201b6540, 0xc4200300a0, 0xa, 0xa, 0x0, 0x0)
    	/go/src/github.com/moby/buildkit/vendor/github.com/urfave/cli/app.go:268 +0x60c
    main.main()
    	/go/src/github.com/moby/buildkit/cmd/buildkitd/main.go:238 +0xc64

Also add some random labels to the integration sandbox (which I have confirmed
is enough to trigger this issue before the fix).

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-09-03 11:35:36 +01:00
Tonis Tiigi c524709f4a buildkitd: add config file support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-30 16:11:55 -07:00
Tonis Tiigi d86efeb63a buildkitd: allow custom worker platforms for containerd
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-16 13:54:59 -07:00
Akihiro Suda 27b6ab4940 rootless: fix default path configuration
if buildkitd is being executed as the mapepd-root ($USER==root)
in a rootless container, we need to enable the rootless mode but
we don't want to honor $HOME.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-07-02 15:47:57 +09:00
Tonis Tiigi a1bafadfd5 worker: add platforms support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-22 11:46:46 -07:00
Akihiro Suda b003c68521 oci-worker: resolve snapshotter label to real name
before: `org.mobyproject.buildkit.worker.snapshotter:auto`
after:  `org.mobyproject.buildkit.worker.snapshotter:overlayfs`

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-06-05 17:38:16 +09:00
Akihiro Suda adef0dedef oci-worker: experimental support for rootless mode
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-05-31 16:05:13 +09:00
Akihiro Suda 126f91dea7 update containerd (binary: v1.0.3, library: Apr 3, 2018)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-04-04 17:22:20 +09:00
Akihiro Suda de73349f01 worker/oci: allow specifying arbitrary snapshotter factory
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-02-26 18:15:10 +09:00
Akihiro Suda 622d4a442a worker/oci: allow using naive snapshotter
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-02-20 12:13:18 +09:00
Akihiro Suda 84c0dd0b07 worker: add labels
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-20 16:28:47 +09:00
Akihiro Suda 2f2b72989c *: buildd -> buildkitd
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-19 13:23:27 +09:00