open-telemetry/opentelemetry-specification#740 has decided to
promote different set of env for CLI propagation.
Switch to use them so we are more consistent with other
tools. Old ones should be removed in a future release.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Before this change, the lower and upper parents provided to the cache
manager Diff method were not cloned, which resulted in some code paths
incorrectly providing them directly as the parents of the returned ref.
This meant that if they were released after the call to Diff, the diff
ref could become incorrectly invalidated.
Now, the lower and upper are cloned and unit test coverage has been
added to test that ref release is handled correctly.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Instead of doing a direct string comparison we should
use `platforms.Only` so that we can also detect the
variants that are compatible but don’t match directly.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Before this, there could be crash during a call to finalize a ref that
occured after the snapshot was committed but before committing the
metadata that indicates the immutable ref no longer had an
equalMutable. This resulted in a situation where any future calls to
finalize that ref would fail.
Now, if that situation happens, the cache will notice when it's
initially loaded that the ref has an equalMutable that's missing its
snapshot and that its own snapshot exists. It will then just use the
correctly committed snapshot and clear the equalMutable field.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This adds test coverage for ensuring the readonly parameter is honored
as expected in the ref Mount methods. There was a regression introduced
during #2335 that went unnoticed until identified and fixed in #2562.
This test coverage should help prevent similar regressions in the
future.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This change enables inline cache to work as expected with MergeOp by
supporting a new type of result, DiffResult, which enables results to be
specified as a specific ordered set of layers inside an image.
Previously, results could only be specified with a singe layer index,
which meant that they had to start at the image's base layer and end at
that index. That meant that merge inputs couldn't be specified as they
are often a subset of the image layers that don't begin at the base.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Before this, you could return worker ref results from ops that have nil
refs but once they were attempted to be used, various nil exceptions
would get hit. Now, those cases should be handled.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This allows you to create refs that are single layers representing the
diff between any two arbitrary refs. The primary use case for this is
to allows users to extract the changes created by ops like Exec and
rebase them elsewhere through MergeOp. However, there is no restriction
on the inputs to DiffOp and the resulting ref's layer is simply the
layer created by running the differ on the two inputs refs
(specifically, the same differ used during exports).
A Diff ref can be mounted by itself, in which case it is defined as the
result of applying the diff to Scratch. Most use cases though will use
Diff refs as the input to a MergeOp, in which case the diff is just
applied on top of the lower merge inputs, as was the case before.
In cases like Diff(A, A->B->C) (i.e. cases where the diff is between two
refs where the lower is an ancestor of upper), the diff will be defined
as the layers separating the two refs. In other cases, the diff is just
a single layer, not re-used from the inputs, representing the diff
between the two refs (which can be defined as the layer "Diff(A,B)" that
satisfies "Merge(A, Diff(A,B)) == B").
Note that there is technically a meaningful difference between the
"unmerge" behavior of extracting the layers separating diffs and the
"simple diff" of just running the differ on the two refs. Namely, in the
case where there are "intermediate deletes" (i.e. deletes that only
exist in layers between A and B but not between A and B by themselves),
then the simple diff and unmerge can create different results when
plugged into a MergeOp. This is due to the fact that intermediate
deletes will apply to the merge when using the unmerge behavior, but not
when using the simple diff. This is on top of the fact that the simple
diff inherently has a "flattening" behavior where multiple layers are
squashed into a single one.
So, in the case where lower is an ancestor of upper, we choose to follow
the unmerge behavior, but it's possible users may prefer the simple diff
behavior. As of right now, they won't be able to do so, but if needed we
can add the ability to choose which behavior is followed in the future.
This could be done through a flag provided to DiffOp or possibly by
adapting llb.Copy to support this type of behavior with the same
efficiency as DiffOp.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
If the ref is invalid we are seeing a panic from `umask-git` because
the error type is not always a unix.WaitStatus:
```
#1 0.227 fatal: Not a valid object name 000111222333444555666777888999aaabbbcccd^{commit}
#1 0.229 panic: interface conversion: interface {} is syscall.WaitStatus, not unix.WaitStatus
#1 0.229
#1 0.229 goroutine 1 [running]:
#1 0.229 github.com/moby/buildkit/source/git.gitMain()
#1 0.229 /src/source/git/gitsource_unix.go:66 +0x27d
#1 0.229 github.com/docker/docker/pkg/reexec.Init(...)
#1 0.229 /src/vendor/github.com/docker/docker/pkg/reexec/reexec.go:26
#1 0.229 main.init.0()
#1 0.229 /src/cmd/buildkitd/main.go:76 +0xf6
#1 0.633 fatal: reference is not a tree: 000111222333444555666777888999aaabbbcccd
#1 0.635 panic: interface conversion: interface {} is syscall.WaitStatus, not unix.WaitStatus
#1 0.635
#1 0.635 goroutine 1 [running]:
#1 0.635 github.com/moby/buildkit/source/git.gitMain()
#1 0.635 /src/source/git/gitsource_unix.go:66 +0x27d
#1 0.635 github.com/docker/docker/pkg/reexec.Init(...)
#1 0.635 /src/vendor/github.com/docker/docker/pkg/reexec/reexec.go:26
#1 0.635 main.init.0()
#1 0.635 /src/cmd/buildkitd/main.go:76 +0xf6
```
This is from trying to solve:
```
llb.Git("https://github.com/moby/buildkit.git", "000111222333444555666777888999aaabbbcccd")
```
Signed-off-by: coryb <cbennett@netflix.com>
Stages and implicit stages from image names can be
redefined with build options.
This enables using more that one source directory
and reusing results from other builds. This can also
be used to use a local image from other build without
including a registry.
Contexts need to be defined as `context:name=` frontend
options. The value can be image, git repository,
URL, local directory or a frontend input.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Tonis Tiigi
CrazyMax
Erik Sipsma
Akihiro Suda
Kohei Tokunaga
Shiwei Zhang
Edgar Lee
coryb
CrazyMax
Christian Weichel