Before this change buildkit was changing the media type for
non-distributable layers to normal layers.
It was also clearing out the urls to get those blobs.
Now the layer mediatype and URL's are preserved.
If a layer blob is seen more than once, if it has extra URL's they will
be appended to the stored value.
On export there is now a new exporter option to preserve the
non-distributable data values.
All URL's seen by buildkit will be added to the exported content.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
The merge diff tests were using some common LLB state objects across
multiple test cases, which each have their own buildkit client and ran
in parallel. The LLB client library does not appear to currently work
with such use cases where LLB states are shared, with errors about
concurrent map writes to marshal state constraints occasionally being
hit during test runs.
Until that issue is addressed (or it's made explicit that the LLB client
library is not expected to work in such a way), this works around the
problem by not sharing states and instead using common funcs that return
distinct state instances.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This fixes an issue where if a vertex rapidly had progress updates (such
as the case where CacheMap runs in succession for multiple indexes),
then progress updates were all using the same ID and could incorrectly
override one another, resulting in missing vertex updates.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Previously, vertexes sent to progress might have multiple intervals in
which they started+stopped, but the output only knew about the most
recent interval. Now, all the intervals are tracked and the total time
spent on each (after accounting for overlaps) is displayed. This is
implemented both for normal vertexes and progress group vertexes.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This allows clients to specify that LLB states should be grouped in
progress output under a custom name. Status updates for all vertexes in
the group will show up under a single vertex in the output.
The intended use cases are for Dockerfile COPY's that use MergeOp as a
backend and for grouping some other internal vertexes during frontend
builds.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Now, when a merge or diff ref is unlazied, the progress will show up
under the vertex for the merge/diff ref. Additionally, any ancestors of
the op that also need to be unlazied as part of unlazying the merge/diff
will show status updates under its vertex in the progress.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
compression-level option can be set on export to
define the preferred speed vs compression ratio. The
value is a number dependent on the compression algorithm.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This adds a couple more integration tests to more fully define the
behavior of these interactions. Additionally, through this, a minor
difference to POSIX was discovered where quotes are supposed to be
properly preserved in a heredoc (since a heredoc is treated as a
double-quoted word).
To handle this, a new property, SkipProcessQuotes is added to the shell
lexer which simply treats quotes as ordinary characters. This is the
only behavioral change needed to actually get the new tests working.
Signed-off-by: Justin Chadwell <me@jedevc.com>
In the contents of COPY/ADD, we perform expansion on variables using a
lexer. However, this lexer, by default, removes quotes as well as
expanding variables - this isn't really the kind of behavior we're
after, as it feels quite unintuitive.
To fix this, we introduce a new ExpandRaw function, which commands can
implement that implement an alternative expansion that preserves quotes
(and possibly other characters/features in the future).
Additionally, we introduce new tests to more clearly define the desired
behavior. One major note is that backslashes are not passed directly,
and are processed, following normal escape rules (so that we can use `$`
symbols directly).
Signed-off-by: Justin Chadwell <me@jedevc.com>
Previously, we'd only write raw escapes out as words, and not out to the
result. We weren't using or relying on this behaviour, but it could
easily have caused a bug if we were. This patch just cleans rawEscapes
to behave like rawQuotes.
Signed-off-by: Justin Chadwell <me@jedevc.com>
This fixes errors such as:
error: cannot update ref 'refs/heads/v2.30.0': trying to write non-commit object 2d9685d47a7e516281aa093bf0cddc8aafa72448 to branch 'refs/heads/v2.30.0'
which occur when cloning a tag rather than branch.
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
open-telemetry/opentelemetry-specification#740 has decided to
promote different set of env for CLI propagation.
Switch to use them so we are more consistent with other
tools. Old ones should be removed in a future release.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Before this change, the lower and upper parents provided to the cache
manager Diff method were not cloned, which resulted in some code paths
incorrectly providing them directly as the parents of the returned ref.
This meant that if they were released after the call to Diff, the diff
ref could become incorrectly invalidated.
Now, the lower and upper are cloned and unit test coverage has been
added to test that ref release is handled correctly.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Brian Goff
Erik Sipsma
Tõnis Tiigi
Aaron Lehmann
Kohei Tokunaga
coryb
Justin Chadwell
CrazyMax
Edgar Lee
Alex Couture-Beil
Akihiro Suda
dependabot[bot]
CrazyMax