Modelled after the vendor support provide a validator and an updator for files
produced by `go generate` (which today just means `*.pb.go`).
Main difference from the vendor support is that we are no longer simply nuking
and replacing a single directory, so I ended up hardcoding `*.pb.go` in a bunch
of places which I don't like but cannot see a way around which doesn't risk
nuking people's other local changes.
The generated files are placed in an unpacked form in a `FROM scratch`
container for update. Use a subdirectory and `tar --strip-components` (portable
to MacOS and Linux according to `tar(1)`) since trying to do a `docker export`
of just the root ends up adding `.dockerenv`, `sys`, `proc` and `dev` to the
source tree.
The validate container is not `FROM scratch` because we want `cat`.
The run in `frontend/gateway/pb/generate.go` was missing an include so fix
that.
The versions of `protoc` and the gogo plugins were chosen to regenerate the
existing code as closely as possible. The updates to `*.pg.go` here are all the
result of regenerating with go1.9 which fixed
https://github.com/golang/go/issues/17663 and replaced an invalid timestamp in
the gzip header of the data encoded in `fileDescriptor*`, and adopted a new
standard for marking generated files.
Finally, I noticed that my `docker run`s were missing an `--rm` which I
inherited from `validate-vendor`, so fix all those.
Closes: #322
Signed-off-by: Ian Campbell <ijc@docker.com>
Otherwise the daemon panics when generating the OCI spec.
For belt and braces check in the ExecOp Run function but also when generating the spec.
Signed-off-by: Ian Campbell <ijc@docker.com>
This seems to have missed out on an update/refactor at some point and was not
usable in its previous form without duplicating the `runOptionFunc`
scaffolding.
Signed-off-by: Ian Campbell <ijc@docker.com>
At the moment vendor.conf is missing entries for `github.com/tonistiigi/llb-gobuild`
and `github.com/morikuni/aec` due in part because of a combination of
https://github.com/LK4D4/vndr/issues/62 and https://github.com/LK4D4/vndr/issues/63.
The issue vndr#63 (related to lack of `github.com/morikuni/aec`) can be worked
around by removing the vendor directory before rerunning `vndr`, so do so.
Due to vndr#62 the issue with `github.com/tonistiigi/llb-gobuild` cannot be
detected at the moment, but pass `-strict` to `vndr` in anticipation of a fix
there.
Signed-off-by: Ian Campbell <ijc@docker.com>
When using `--frontend=gateway.v0` in order to use a tagged image as the
frontend it is currently required to push the tag to a registry, so that
buildkit can pull it. One can use the `image@sha256:digest` form but this is
inconvenient for the development cycle.
Instead, introduce a proxy `Resolver` which if the pull fails checks the local
image store (if there is one, which is worker dependent) for a suitable image
and if so uses that.
With this I can now:
buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=. --exporter=image --exporter-opt=name=docker.io/ijc25/fe:dev
builtctl build --frontend=gateway.v0 --frontend-opt=source=ijc25/fe:dev --exporter=image --exporter-opt=name=hello-world:dev
Without any push to a registry.
I am using the containerd worker.
Signed-off-by: Ian Campbell <ijc@docker.com>
The previous code had 3 issues:
- The original schema 1 manifest remained in `ongoing.added` but was not added
to the content store by the conversion. #296 tried to address this in an
incorrect way and was reverted by the previous commit. Instead switch in the
schema2 manifest, which was otherwise missed.
- Empty layers in the schema 1 manifest are not propagated to the schema 2
version, meaning they are not referenced by anything and hence are never
freed up and therefore leak
- The new schema 2 image config is generated by the converter and not fetched,
so it never passes through the `HandlerFunc` which adds things to
`ongoing.added`.
Two address these last two bullets when processing a schema 1 image walk over
the converted image and together with `ongoing.added` produce lists of used and
unused blobs, use the list of used blobs to perform the GC updates and simply
delete everything on the unused list.
Closes: #301.
Signed-off-by: Ian Campbell <ijc@docker.com>
This reverts commit 92101de156 from #296.
This was the wrong approach a subsequent commit will fix this properly.
Signed-off-by: Ian Campbell <ijc@docker.com>
Tõnis Tiigi
Akihiro Suda
Ian Campbell
Vincent Demeester
Jess Frazelle
Akihiro Suda