When server returns a weak etag, the prefix can
cause a mismatch on direct comparison with old value.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This allows clients to specify that LLB states should be grouped in
progress output under a custom name. Status updates for all vertexes in
the group will show up under a single vertex in the output.
The intended use cases are for Dockerfile COPY's that use MergeOp as a
backend and for grouping some other internal vertexes during frontend
builds.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Now, when a merge or diff ref is unlazied, the progress will show up
under the vertex for the merge/diff ref. Additionally, any ancestors of
the op that also need to be unlazied as part of unlazying the merge/diff
will show status updates under its vertex in the progress.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This fixes errors such as:
error: cannot update ref 'refs/heads/v2.30.0': trying to write non-commit object 2d9685d47a7e516281aa093bf0cddc8aafa72448 to branch 'refs/heads/v2.30.0'
which occur when cloning a tag rather than branch.
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
If the ref is invalid we are seeing a panic from `umask-git` because
the error type is not always a unix.WaitStatus:
```
#1 0.227 fatal: Not a valid object name 000111222333444555666777888999aaabbbcccd^{commit}
#1 0.229 panic: interface conversion: interface {} is syscall.WaitStatus, not unix.WaitStatus
#1 0.229
#1 0.229 goroutine 1 [running]:
#1 0.229 github.com/moby/buildkit/source/git.gitMain()
#1 0.229 /src/source/git/gitsource_unix.go:66 +0x27d
#1 0.229 github.com/docker/docker/pkg/reexec.Init(...)
#1 0.229 /src/vendor/github.com/docker/docker/pkg/reexec/reexec.go:26
#1 0.229 main.init.0()
#1 0.229 /src/cmd/buildkitd/main.go:76 +0xf6
#1 0.633 fatal: reference is not a tree: 000111222333444555666777888999aaabbbcccd
#1 0.635 panic: interface conversion: interface {} is syscall.WaitStatus, not unix.WaitStatus
#1 0.635
#1 0.635 goroutine 1 [running]:
#1 0.635 github.com/moby/buildkit/source/git.gitMain()
#1 0.635 /src/source/git/gitsource_unix.go:66 +0x27d
#1 0.635 github.com/docker/docker/pkg/reexec.Init(...)
#1 0.635 /src/vendor/github.com/docker/docker/pkg/reexec/reexec.go:26
#1 0.635 main.init.0()
#1 0.635 /src/cmd/buildkitd/main.go:76 +0xf6
```
This is from trying to solve:
```
llb.Git("https://github.com/moby/buildkit.git", "000111222333444555666777888999aaabbbcccd")
```
Signed-off-by: coryb <cbennett@netflix.com>
This consists of just the base MergeOp with support for merging LLB
results that include deletions using hardlinks as the efficient path
and copies as fallback.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This is mostly just preparation for merge-op. The existing
Extract method is updated to be usable for unlazying any type of refs
rather than just lazy blobs. The way views are created is simplified and
centralized in one location.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
The Parent method will no longer make sense with forthcoming Merge and
Diff support as refs will become capable of having multiple parents. It
was also only ever used externally to get the full chain of refs for
each layer in the ref's chain.
The newly added LayerChain method replaces Parents with a method that
just returns a slice of refs for each layer in the ref's chain. This
will work more seamlessly with Merge and Diff (in which case it returns
the "flattened" ancestors of the ref) in addition to being a bit easier
to use for the exiting cases anyways.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
There are a few goals with this refactor:
1. Remove external access to fields that no longer make sense and/or
won't make sense soon due to other potential changes. For example,
there can now be multiple blobs associated with a ref (for different
compression types), so the fact that you could access the "Blob"
field from the Info method on Ref incorrectly implied there was just
a single blob for the ref. This is on top of the fact that there is
no need for external access to blob digests.
2. Centralize use of cache metadata inside the cache package.
Previously, many parts of the code outside the cache package could
obtain the bolt storage item for any ref and read/write it directly.
This made it hard to understand what fields are used and when. Now,
the Metadata method has been removed from the Ref interface and
replaced with getters+setters for metadata fields we want to expose
outside the package, which makes it much easier to track and
understand. Similar changes have been made to the metadata search
interface.
3. Use a consistent getter+setter interface for metadata, replacing
the mix of interfaces like Metadata(), Size(), Info() and other
inconsistencies.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
This is a safer alternative until we figure out why
http.Transport based limiting fails.
Some connections like cache export/import do not have a
domain key atm and these connections use global pool.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Unfortunately, further test cases will have to be removed because gitindentifier will now leave the branch blank instead of filling it in
Signed-off-by: Levi Harrison <levisamuelharrison@gmail.com>
It is my suspecion that the tests were failing on previous commits because of the lack of authentication and other stuff like that available in gitidentifier as compared to gitsource
Signed-off-by: Levi Harrison <levisamuelharrison@gmail.com>
Instead of just assuming that the default branch is master, use ls-remote to find out. Also removed tests that didn't specifiy a branch but required authentication, because those will fail now that the repo is actually checked.
Signed-off-by: Levi Harrison <levisamuelharrison@gmail.com>
This updates all occurrences of Go 1.13 to Go 1.16; also updated
the code that's used to redact credentials in URLs to use the Go
implementation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- this allows one to use a non-standard ssh username like repo@host
rather than assuming it must be git@host
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
Tonis Tiigi
Erik Sipsma
Edgar Lee
Alex Couture-Beil
coryb
CrazyMax
Kohei Tokunaga
Koichi Shiraishi
Morlay
Levi Harrison
Akihiro Suda
Sebastiaan van Stijn
Pierre Fenoll