Commit Graph

58 Commits (a6fc85f163909a437249828d8ff6f63001601079)

Author SHA1 Message Date
Erik Sipsma 9321ec2f82 Refactor cache record mount cache.
This is mostly just preparation for merge-op. The existing
Extract method is updated to be usable for unlazying any type of refs
rather than just lazy blobs. The way views are created is simplified and
centralized in one location.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-11-17 11:02:16 -08:00
CrazyMax 54b8ff2fc8
go fmt: add //go:build
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-28 13:26:43 +02:00
Erik Sipsma a9f1980ebb Refactor cache metadata interface.
There are a few goals with this refactor:
1. Remove external access to fields that no longer make sense and/or
   won't make sense soon due to other potential changes. For example,
   there can now be multiple blobs associated with a ref (for different
   compression types), so the fact that you could access the "Blob"
   field from the Info method on Ref incorrectly implied there was just
   a single blob for the ref. This is on top of the fact that there is
   no need for external access to blob digests.
2. Centralize use of cache metadata inside the cache package.
   Previously, many parts of the code outside the cache package could
   obtain the bolt storage item for any ref and read/write it directly.
   This made it hard to understand what fields are used and when. Now,
   the Metadata method has been removed from the Ref interface and
   replaced with getters+setters for metadata fields we want to expose
   outside the package, which makes it much easier to track and
   understand. Similar changes have been made to the metadata search
   interface.
3. Use a consistent getter+setter interface for metadata, replacing
   the mix of interfaces like Metadata(), Size(), Info() and other
   inconsistencies.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-08-25 19:15:09 +00:00
Koichi Shiraishi 671ae38fb7
all: unify the specs-go package import alias to ocispecs
ocispecs means "O"pen "C"ontainer "I"nitiative image-spec/"specs"-go/v1
                      opencontainers          /image-spec/specs-go/v1

Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
2021-08-11 08:29:09 +09:00
Morlay 18b49fd7dc refactor to use util/bklog instead of using logurs directly
Signed-off-by: Morlay <morlay.null@gmail.com>
2021-07-13 11:42:31 +08:00
Tonis Tiigi d512920c24 support collecting traces from llb.Exec
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-20 21:57:35 -07:00
Vlad A. Ionescu b3cf7c43cf Switch to Acquire API.
Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>
2021-05-12 13:41:33 +03:00
Brian Goff d9834f872c
Add support for apparmor/selinux
Set's an apparmor profile on the OCI spec if one is configred on the
worker.
Adds selinux labels to containers (only added if selinux is enabled on
the system).

This assumes that the specified apparmor profile is already loaded on
the system and does not try to load it or even check if it is loaded.

SELinux support requires the `selinux` build tag to be added.
Likewise, `runc` would require both the `apparmor` and `selinux` build
tags.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>

Vendored go-selinux to v1.8.0
Fixed tests

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 68bb095353)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-04 22:01:34 +01:00
Tonis Tiigi 970e630f1b add debug for runcworker test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-13 18:44:54 -08:00
Tonis Tiigi bdcee17437 executor: change mount to struct
Allows readonly passed cleanly.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:51 -08:00
Tonis Tiigi e3b05289d8 add session injection to remote loading
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:44 -08:00
Cory Bennett 6a56695d3a remove `Get` prefix from Executor/CacheManager funcs and update to avoid naming conflicts
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-17 00:54:15 +00:00
Cory Bennett 5e91dff4ed fix error handling for exec when container fails to start
update run/exec tests for stdin and expected failures
move common tests for runc and container to shared tests package

Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-10 22:06:42 +00:00
Cory Bennett 6d58121c11 Update Executor interface for Run and Exec
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-09 23:40:36 +00:00
Tonis Tiigi 2e9987ad16 session: track sessions with a group construct
Avoid hidden session passing and allow one session to drop when
multiple builds share a vertex.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-05 00:15:25 -07:00
Nikhil Pandeti 7efcee49c5 Oci worker binary documentation and add missing constructor arg in runc_test
Signed-off-by: Nikhil Pandeti <nikhil.pandeti@utexas.edu>
2019-12-12 13:25:56 -06:00
Jeffrey Huang 20258652b3 Fix issue #1230
To support switching the OCI worker binary via buildkitd and without modifying the Dockerfile, an optional command-line flag was added to main_oci_worker.go that allowed for input the name of an OCI worker binary (ex. crun). This OCI worker binary would then replace the current buildkit-runc with a symbolic link to that binary. The above was done using the os/exec package's Command func.

Signed-off-by: Jeffrey Huang <jeffreyhuang23@gmail.com>
2019-12-12 04:33:24 -06:00
Tonis Tiigi 31a9aeea88 cache: add migration flow to new lease based format
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-10-16 10:35:49 -07:00
Tonis Tiigi 688e2c2272 cache: update components to new lease based cache manager
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-10-16 10:32:04 -07:00
Derek McGowan d5108d038d Update to containerd 1.3.0
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-09-30 14:45:18 -07:00
Tonis Tiigi 57d19b1208 network: refactor cni support to multiple packages
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-06 12:52:37 -07:00
Tonis Tiigi e7759a861b client: add cni network tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-10 18:00:06 -07:00
Tonis Tiigi bc7a6f2556 add cni networking support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-10 14:42:30 -07:00
Tibor Vass 8e692507ee executor: allow setting DNSConfig to be used by resolvconf
This patch allows downstream code to pass a DNSConfig that is
then used by executor/oci.GetResolvConf.

This would allow the BuildKit-based builder in Docker to honor
the docker daemon's DNS configuration, thus fixing a feature gap
with the legacy builder.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-17 20:43:33 +00:00
Tonis Tiigi 8e25187cc7 pull: fix content blobs deletion on pull race
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-05-28 14:39:05 -07:00
Wei Fu ffad8ada79 exporter: support unpack opt for image exporter
It is enhancement which allows to unpack image into containerd
snapshotter storage by `--output type=image,<.>=<.>,unpack=true`.

In order to support this feature, we needs to extend the Snapshotter
witwh `Name() string` function. Because we needs to set gc label for
snapshotter which need snapshotter name.

fix: #908

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-04-24 21:24:06 +08:00
Tonis Tiigi 894da35fed fix unit tests for idmapping support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-04-02 20:35:18 -07:00
Tonis Tiigi 9f53ea3d78 userns support for sources and executor
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-02 18:26:04 -07:00
Tonis Tiigi 6921dbe0f6 snapshot: base for userns remapping
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-02 18:23:46 -07:00
Akihiro Suda 26010a4864 decouple SessionManager from Worker
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-02-23 22:55:17 +09:00
Akihiro Suda c54f4a986d support --oci-worker-no-process-sandbox
Note that this mode allows build executor containers to kill (and potentially ptrace) an arbitrary process in the BuildKit host namespace.
This mode should be enabled only when the BuildKit is running in a container as an unprivileged user.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-01-08 10:42:52 +09:00
Tonis Tiigi ddfb7f77c9 hack: update test dockerfiles to buildkit
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-10 23:26:05 -07:00
John Howard 2de2c04c8e Revendoring to move boltdb to bbolt
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-09-18 11:18:08 -07:00
Michael Crosby b97bc71adb Refactor networking with ns paths
This fixes the issues where buildkit and callers do not have to be a
subpreaper in order to use networking.  I can add CNI provider later,
with a hidden sub command to create a new network namespace and bind
mount it to buildkit's state dir.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-21 13:37:47 -04:00
Tonis Tiigi f8dd602282 runc: improve canceling
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-07 11:51:56 -07:00
Kunal Kushwaha 765f1b64b9 executor: allow network providers
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-07 11:51:56 -07:00
Tonis Tiigi bc765861be diff: implement windows layer support for linux
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-16 16:33:21 -07:00
Tonis Tiigi a1bafadfd5 worker: add platforms support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-22 11:46:46 -07:00
Kunal Kushwaha e991a846be Default mounts set for buildkit
containerd/oci_specs mounts /run, which is not
required by buildkit.

Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-06-07 14:43:44 +09:00
Akihiro Suda adef0dedef oci-worker: experimental support for rootless mode
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-05-31 16:05:13 +09:00
Tonis Tiigi e24e194078 remove net/context references
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-30 18:10:54 -07:00
Tonis Tiigi 2dba0488c8 runc: make command name configurable
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-17 20:39:19 -07:00
Tõnis Tiigi 89f8bc1c02
Merge pull request #329 from AkihiroSuda/containerd-20180403
update containerd (binary: v1.0.3, library: Apr 3, 2018)
2018-04-04 10:05:42 -07:00
Akihiro Suda 126f91dea7 update containerd (binary: v1.0.3, library: Apr 3, 2018)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-04-04 17:22:20 +09:00
Ian Campbell 4bae684e32 Invert tags on runc worker test so they run by default.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 11:01:46 +01:00
Tonis Tiigi da633a8e6b worker: throttle calls to gc
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-19 10:40:31 -07:00
Akihiro Suda de73349f01 worker/oci: allow specifying arbitrary snapshotter factory
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-02-26 18:15:10 +09:00
Akihiro Suda 622d4a442a worker/oci: allow using naive snapshotter
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-02-20 12:13:18 +09:00
Tonis Tiigi 7906fb3085 vendor: update containerd to v1.0.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-13 16:00:09 -08:00
Tonis Tiigi b0679c66db snapshot: clean up snapshot interface
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-04 23:09:05 -08:00