Before this, there was a bug triggered under the following conditions:
1. An overlay snapshotter was being used, which caused the optimization
of preparing a new snapshot off of the base layers to be triggered
2. The base layers contained a directory that had contents
3. One subsequent layer deleted that directory w/out recreating it
4. A later layer recreated the directory
In this case, what happened was a whiteout device would be created as
part of 3 above but then in step 4 the whiteout device would be removed
and replaced with a plain directory. The problem is that such a
directory doesn't block out the files from step 2 and it doesn't know
about them because they are in a lowerdir (not the upperdir being
applied to).
The simplest fix, which this commit implements, is to just set the
directory created in step 4 as opaque, which enables the correct
behavior of blocking out files below it.
This was missed in test coverage before because tests for opaque
handling always combined 3+4 into one layer, whereas the bug requires
they be separate layers. A new integration test has been added to cover
this case.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
The merge diff tests were using some common LLB state objects across
multiple test cases, which each have their own buildkit client and ran
in parallel. The LLB client library does not appear to currently work
with such use cases where LLB states are shared, with errors about
concurrent map writes to marshal state constraints occasionally being
hit during test runs.
Until that issue is addressed (or it's made explicit that the LLB client
library is not expected to work in such a way), this works around the
problem by not sharing states and instead using common funcs that return
distinct state instances.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This allows clients to specify that LLB states should be grouped in
progress output under a custom name. Status updates for all vertexes in
the group will show up under a single vertex in the output.
The intended use cases are for Dockerfile COPY's that use MergeOp as a
backend and for grouping some other internal vertexes during frontend
builds.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This change enables inline cache to work as expected with MergeOp by
supporting a new type of result, DiffResult, which enables results to be
specified as a specific ordered set of layers inside an image.
Previously, results could only be specified with a singe layer index,
which meant that they had to start at the image's base layer and end at
that index. That meant that merge inputs couldn't be specified as they
are often a subset of the image layers that don't begin at the base.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
HTTP/2(RFC7540) defines :authority pseudo header includes the authority portion
of target URI but it must not include userinfo part (i.e. url.Host).
However, when TLS certificate specified, grpc-go requires it must match
with its servername specified for certificate validation.
Signed-off-by: Shingo Omura <everpeace@gmail.com>
Using an interface instead of a func is more flexible while achieving
the same effect. It allows you to succintly define a large number of
test cases as structs, as is common in table-driven testing.
A helper func is added that converts the existing test funcs into the
interface, so the change is fairly seamless.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
When vendored into moby, the local exporter uses a user map that results
in all files being given at most 755 permissions. This change updates
the test to use permissions less than or equal to that to make tests
equivalent whether running w/ dockerd worker or any other type.
Additionally, add assertions that exported images also have the expected
contents, which helps ensures consistency between images created by
dockerd and those created by vanilla buildkit.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Before this change, if the path provided to FileOp.Rm was a symlink then
the target of the symlink would be removed instead of the symlink
itself. Now, the symlink will be removed instead. However, any symlinks
present in the parent dirs of the specified path will still be resolved
before calling os.Remove; this change only results in the base of the
specified path not being followed.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This consists of just the base MergeOp with support for merging LLB
results that include deletions using hardlinks as the efficient path
and copies as fallback.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
The problem this change is trying to fix are the environments where some
capabilities are already dropped, so they can't be granted to the
job with `--security=insecure`.
I know that probably fixed set of capabilities was implemented to
provide a stable build environment, but at the same time this breaks
environments with reduced capabilities.
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
Erik Sipsma
Tõnis Tiigi
Aaron Lehmann
Cory Bennett
CrazyMax
Akihiro Suda
Kohei Tokunaga
Shingo Omura
Andrey Smirnov
Koichi Shiraishi