This just makes sure the logic for the layer conversion is all in one
place and settable by a common option.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Now, when a merge or diff ref is unlazied, the progress will show up
under the vertex for the merge/diff ref. Additionally, any ancestors of
the op that also need to be unlazied as part of unlazying the merge/diff
will show status updates under its vertex in the progress.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
compression-level option can be set on export to
define the preferred speed vs compression ratio. The
value is a number dependent on the compression algorithm.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Working with strings is error-prone because a platform
can be in multiple string forms and less flexible.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Before this, you could return worker ref results from ops that have nil
refs but once they were attempted to be used, various nil exceptions
would get hit. Now, those cases should be handled.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Using an interface instead of a func is more flexible while achieving
the same effect. It allows you to succintly define a large number of
test cases as structs, as is common in table-driven testing.
A helper func is added that converts the existing test funcs into the
interface, so the change is fairly seamless.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This consists of just the base MergeOp with support for merging LLB
results that include deletions using hardlinks as the efficient path
and copies as fallback.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This is mostly just preparation for merge-op. The existing
Extract method is updated to be usable for unlazying any type of refs
rather than just lazy blobs. The way views are created is simplified and
centralized in one location.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Before this, descriptor handlers were not included with calls to the
exporter, which then sometimes called LoadRef and failed to get a ref
because it was lazy. This change results in the DescHandlers of the
already loaded refs to get plugged into context so they can be re-used
by the exporter if it needs to load the ref again.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
FromRemote now calls CheckDescriptor to validate
if the blob still exists. Otherwise cache loading
fallback does not get triggered because cache is
actually lazily pulled in only on exporting phase.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
There are a few goals with this refactor:
1. Remove external access to fields that no longer make sense and/or
won't make sense soon due to other potential changes. For example,
there can now be multiple blobs associated with a ref (for different
compression types), so the fact that you could access the "Blob"
field from the Info method on Ref incorrectly implied there was just
a single blob for the ref. This is on top of the fact that there is
no need for external access to blob digests.
2. Centralize use of cache metadata inside the cache package.
Previously, many parts of the code outside the cache package could
obtain the bolt storage item for any ref and read/write it directly.
This made it hard to understand what fields are used and when. Now,
the Metadata method has been removed from the Ref interface and
replaced with getters+setters for metadata fields we want to expose
outside the package, which makes it much easier to track and
understand. Similar changes have been made to the metadata search
interface.
3. Use a consistent getter+setter interface for metadata, replacing
the mix of interfaces like Metadata(), Size(), Info() and other
inconsistencies.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
e.g.,
```
Labels:
org.mobyproject.buildkit.worker.containerd.namespace: buildkit
org.mobyproject.buildkit.worker.containerd.uuid: 343cfb49-cce9-453f-b21c-e5d22ca2cb47
org.mobyproject.buildkit.worker.executor: containerd
org.mobyproject.buildkit.worker.hostname: suda-ws01
org.mobyproject.buildkit.worker.snapshotter: overlayfs
```
Planned to be used by nerdctl to detect whether containerd exporter can be used
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Set's an apparmor profile on the OCI spec if one is configred on the
worker.
Adds selinux labels to containers (only added if selinux is enabled on
the system).
This assumes that the specified apparmor profile is already loaded on
the system and does not try to load it or even check if it is loaded.
SELinux support requires the `selinux` build tag to be added.
Likewise, `runc` would require both the `apparmor` and `selinux` build
tags.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Vendored go-selinux to v1.8.0
Fixed tests
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 68bb095353)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Plumb default worker by adding GetDefault() to frontend.WorkerInfos
- To avoid cyclic dependency, refactor frontend.WorkerInfos to worker.Infos
- Refactor gateway.NewContainer to share code with llbsolver/ops/exec.go
Signed-off-by: Edgar Lee <edgarl@netflix.com>
This fixes panics in the moby integration of buildkit when using cache.
Panics come from nil Differ in computeBlobChain which is called in GetRemote().
GetRemote() got moved from Worker to ImmutableRef during the lazy-pull refactor.
However, the ability to easily override GetRemote() got lost with that refactor.
This patch attempts to allow for the override while keeping changes minimal.
Signed-off-by: Tibor Vass <tibor@docker.com>
This reverts commit 9290c15ffc.
This was pinned during the upgrade to containerd 1.3 series, which
changed the default runtime on Linux to io.containerd.runc.v2.
No specific rationale was listed for this pinning, and clearly it's the
wrong thing to do in the presence of Windows, which does not have this
runtime.
Instead, we rely on the containerd-internal defaults, which distinguish
the runtimes for Linux and Windows.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Tõnis Tiigi
Brian Goff
Erik Sipsma
CrazyMax
Kohei Tokunaga
Koichi Shiraishi
Morlay
Akihiro Suda
Cory Bennett
Vlad A. Ionescu
Akihiro Suda
Edgar Lee
Tibor Vass
Paul "TBBle" Hampson