This consists of just the base MergeOp with support for merging LLB
results that include deletions using hardlinks as the efficient path
and copies as fallback.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This is mostly just preparation for merge-op. The existing
Extract method is updated to be usable for unlazying any type of refs
rather than just lazy blobs. The way views are created is simplified and
centralized in one location.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Before this, descriptor handlers were not included with calls to the
exporter, which then sometimes called LoadRef and failed to get a ref
because it was lazy. This change results in the DescHandlers of the
already loaded refs to get plugged into context so they can be re-used
by the exporter if it needs to load the ref again.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
There are a few goals with this refactor:
1. Remove external access to fields that no longer make sense and/or
won't make sense soon due to other potential changes. For example,
there can now be multiple blobs associated with a ref (for different
compression types), so the fact that you could access the "Blob"
field from the Info method on Ref incorrectly implied there was just
a single blob for the ref. This is on top of the fact that there is
no need for external access to blob digests.
2. Centralize use of cache metadata inside the cache package.
Previously, many parts of the code outside the cache package could
obtain the bolt storage item for any ref and read/write it directly.
This made it hard to understand what fields are used and when. Now,
the Metadata method has been removed from the Ref interface and
replaced with getters+setters for metadata fields we want to expose
outside the package, which makes it much easier to track and
understand. Similar changes have been made to the metadata search
interface.
3. Use a consistent getter+setter interface for metadata, replacing
the mix of interfaces like Metadata(), Size(), Info() and other
inconsistencies.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
When running a WORKDIR instruction, buildkit will create that folder
and chown it to the currently set user. For this, it will try to read
the /etc/passwd file to get the proper UID, and if that user is not
found in the file, the root user will be considered as the owner.
However, Windows image do not have that file, which will result in
an error while building the image. We can consider not finding
the /etc/passwd file as the same as not finding the user in the file,
which would solve this issue.
Signed-off-by: Claudiu Belu <cbelu@cloudbasesolutions.com>
Finalize was only used outside the cache package in one place, which
called it with the commit arg set to false. The code path followed
when commit==false turned out to essentially be a no-op because
it set "retain cache" to true if it was already set to true.
It was thus safe to remove the only external call to it and remove it
from the interface. This should be helpful for future efforts to
simplify the equal{Mutable,Immutable} fields in cacheRecord, which exist
due to the "lazy commit" feature that Finalize is tied into.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Allow include and exclude patterns to be specified for the "copy" op,
similarly to "local".
Depends on https://github.com/tonistiigi/fsutil/pull/101
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
Relates to a82fff6377/docs/packages.md (proxies)
> (..) the first four of these are the standard built-in build-arg options
> available for `docker build`
> (..) The last, `all_proxy`, is a standard var used for socks proxying. Since
> it is not built into `docker build`, if you want to use it, you will need to
> add the following line to the dockerfile:
>
> ARG all_proxy
Given the we support all other commonly known proxy env-vars by default, it makes
sense to add this one as well.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
A bug in cloneExecOp prevented mounts from being included in the digest
computed for the execOp cache map. This could lead to an exec being
wrongly cached when a different mount was used for a subsequent
execution.
Repro case:
https://gist.github.com/aaronlehmann/cfeaefc028df8127fb85b9b5f9125f2d
In this example, pass2 should generate an empty diff because the /from
and /to mounts are the same busybox image. But before the fix, it uses
the cached result from pass1 (with different mounts) instead.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
I saw these, and there was no comment in the code explaining if there was a reason
for this; these were added in 2be999ba52, but couldn't
find a mention about these particular lines in the review comments, and I couldn't
think of a special reason for it ':-)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Erik Sipsma
CrazyMax
Akihiro Suda
Tonis Tiigi
Kohei Tokunaga
Cory Bennett
Koichi Shiraishi
Claudiu Belu
Morlay
Sebastiaan van Stijn
Aaron Lehmann
Edgar Lee
Vlad A. Ionescu