Commit Graph

737 Commits (69097924af7208207e2ec96207416f14a0f80130)

Author SHA1 Message Date
Tõnis Tiigi 9f8547022b
Merge pull request #319 from ijc/check-mount-source
Check that mounts have a Source when generating the OCI spec
2018-03-22 08:53:28 -07:00
Ian Campbell ac6598d255 Check that mounts always have a target.
Otherwise the daemon panics when generating the OCI spec.

For belt and braces check in the ExecOp Run function but also when generating the spec.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-22 12:05:12 +00:00
Tõnis Tiigi a1af6d9037
Merge pull request #318 from jessfraz/all-changes
export cache
2018-03-21 10:10:18 -07:00
Jess Frazelle b0f34cb8cd
export cache
Signed-off-by: Jess Frazelle <acidburn@microsoft.com>
2018-03-21 12:34:05 -04:00
Tõnis Tiigi 3e90755493
Merge pull request #316 from ijc/readonly-rootfs
Convert ReadonlyRootFS to a RunOption
2018-03-20 10:17:57 -07:00
Ian Campbell 7be9ae6954 Convert ReadonlyRootFS to a RunOption
This seems to have missed out on an update/refactor at some point and was not
usable in its previous form without duplicating the `runOptionFunc`
scaffolding.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-20 16:58:05 +00:00
Akihiro Suda 67baec79bd
Merge pull request #311 from tonistiigi/optimize-marshal
client: speed up llb marshal
2018-03-20 12:21:56 +09:00
Akihiro Suda f635b18790
Merge pull request #314 from tonistiigi/cancel-fix
solver: don’t cancel child pipes from state change
2018-03-20 12:21:33 +09:00
Akihiro Suda b99388e9dd
Merge pull request #315 from tonistiigi/example-fix
examples: fix gobuild example
2018-03-20 12:21:05 +09:00
Tonis Tiigi 8fd2d5431b examples: fix gobuild example
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-19 11:52:52 -07:00
Tonis Tiigi 5717a4f49f solver: don’t cancel child pipes from state change
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-19 11:26:46 -07:00
Tonis Tiigi da633a8e6b worker: throttle calls to gc
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-19 10:40:31 -07:00
Tonis Tiigi 432c191e81 client: speed up llb marshal
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-19 10:25:58 -07:00
Akihiro Suda e37d938d94
Merge pull request #310 from tonistiigi/faster-load
solver: faster vertex loading
2018-03-19 14:38:20 +09:00
Akihiro Suda d8e533b27b
Merge pull request #313 from tonistiigi/cache-interface
solver: simplify cache storage backend
2018-03-19 14:36:07 +09:00
Akihiro Suda bc7501291c
Merge pull request #307 from tonistiigi/solver-query
solver-next: add support for cache tracking through query
2018-03-19 13:26:20 +09:00
Tonis Tiigi 4df3b2e000 solver: simplify cache backend interface
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-18 21:25:55 -07:00
Tonis Tiigi 5593bb9f48 solver: optimize cache storage interface
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-18 21:25:47 -07:00
Tonis Tiigi 0d536d40b9 solver: faster vertex loading
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-18 20:56:49 -07:00
Tonis Tiigi 513018806b solver: add support for cache tracking through query
Fix result releasing bugs.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-15 15:33:56 -07:00
Akihiro Suda 12198eea27
Merge pull request #306 from tonistiigi/contenthash-fix
contenthash: fix ignored files from path separator
2018-03-14 12:11:44 +09:00
Tonis Tiigi 8f8221d173 contenthash: fix ignored files from path separator
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-13 14:40:27 -07:00
Tõnis Tiigi b403f71a62
Merge pull request #304 from tonistiigi/typo-fix
gateway: typo fix
2018-03-08 10:29:46 -08:00
Tonis Tiigi 12fd7edc24 gateway: typo fix
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-08 10:13:26 -08:00
Tõnis Tiigi abe6627919
Merge pull request #303 from ijc/vndr-fixes
vendoring: try to catch more vendoring issues & fixup an existing one
2018-03-08 10:03:52 -08:00
Ian Campbell 86912f507e vendor.conf: Readd github.com/morikuni/aec
Accidentally removed in faf258dd2e ("vendor: add go-units").

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-08 14:46:28 +00:00
Ian Campbell 0d34cceeb6 vendoring: try to catch more vendoring issues.
At the moment vendor.conf is missing entries for `github.com/tonistiigi/llb-gobuild`
and `github.com/morikuni/aec` due in part because of a combination of
https://github.com/LK4D4/vndr/issues/62 and https://github.com/LK4D4/vndr/issues/63.

The issue vndr#63 (related to lack of `github.com/morikuni/aec`) can be worked
around by removing the vendor directory before rerunning `vndr`, so do so.

Due to vndr#62 the issue with `github.com/tonistiigi/llb-gobuild` cannot be
detected at the moment, but pass `-strict` to `vndr` in anticipation of a fix
there.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-08 14:46:28 +00:00
Tõnis Tiigi e8dbd3aeb4
Merge pull request #302 from ijc/improved-schema1-support
Improved schema1 support
2018-03-07 09:41:34 -08:00
Ian Campbell ae0293c4bf Add an integration test for pulling schema1 images
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-07 15:56:22 +00:00
Tõnis Tiigi 18280e73cc
Merge pull request #289 from ijc/allow-local-image-for-gateway-fe
Use local images if pull fails
2018-03-06 09:59:17 -08:00
Ian Campbell ac5091c10b Use local images if pull fails
When using `--frontend=gateway.v0` in order to use a tagged image as the
frontend it is currently required to push the tag to a registry, so that
buildkit can pull it. One can use the `image@sha256:digest` form but this is
inconvenient for the development cycle.

Instead, introduce a proxy `Resolver` which if the pull fails checks the local
image store (if there is one, which is worker dependent) for a suitable image
and if so uses that.

With this I can now:

    buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=. --exporter=image --exporter-opt=name=docker.io/ijc25/fe:dev
    builtctl build --frontend=gateway.v0 --frontend-opt=source=ijc25/fe:dev --exporter=image --exporter-opt=name=hello-world:dev

Without any push to a registry.

I am using the containerd worker.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-06 12:19:34 +00:00
Ian Campbell b7632ad036 Correct content-store management after schema1 conversion
The previous code had 3 issues:

- The original schema 1 manifest remained in `ongoing.added` but was not added
  to the content store by the conversion. #296 tried to address this in an
  incorrect way and was reverted by the previous commit. Instead switch in the
  schema2 manifest, which was otherwise missed.
- Empty layers in the schema 1 manifest are not propagated to the schema 2
  version, meaning they are not referenced by anything and hence are never
  freed up and therefore leak
- The new schema 2 image config is generated by the converter and not fetched,
  so it never passes through the `HandlerFunc` which adds things to
  `ongoing.added`.

Two address these last two bullets when processing a schema 1 image walk over
the converted image and together with `ongoing.added` produce lists of used and
unused blobs, use the list of used blobs to perform the GC updates and simply
delete everything on the unused list.

Closes: #301.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-02 16:26:47 +00:00
Ian Campbell 0ff79ec71c Revert "Swallow a NotFound error when deleting content, to support schema 1 manifests"
This reverts commit 92101de156 from #296.

This was the wrong approach a subsequent commit will fix this properly.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-02 16:10:19 +00:00
Akihiro Suda 1b31410147
Merge pull request #299 from tonistiigi/cache-release
solver: add support for releasing cached results
2018-02-28 15:46:19 +09:00
Tonis Tiigi 832c140d4a solver: add release support for bolt backend
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-27 15:54:21 -08:00
Tonis Tiigi a9db799188 solver: add support for releasing cached results
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-27 15:53:18 -08:00
Tõnis Tiigi 666a5e4119
Merge pull request #293 from AkihiroSuda/move-llb
llb: update docs
2018-02-27 07:42:03 -08:00
Akihiro Suda ecf2e8dbc9 llb: update docs
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-02-27 16:06:01 +09:00
Akihiro Suda c5795fea02
Merge pull request #297 from tonistiigi/boltdb-cache
solver: add boltdb cache store and test suite
2018-02-27 12:48:56 +09:00
Tonis Tiigi e8af448b3a solver: add boltdb cache store and test suite
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-26 17:42:10 -08:00
Tõnis Tiigi cd6e788437
Merge pull request #296 from ijc/support-schema1-manifest
Swallow a NotFound error when deleting content, to support schema 1 m…
2018-02-26 11:31:58 -08:00
Ian Campbell 92101de156 Swallow a NotFound error when deleting content, to support schema 1 manifests
The behaviour of `github.com/containerd/containerd/remotes/docker/schema1` is
such that the manifest is not actually in the content store, so attempting to
delete it fails with `NotFound`, which with the `gateway.v0` frontend results
in:

    time="2018-02-26T17:01:15Z" level=error msg="fatal error: rpc error: code = Unknown desc = content digest sha256:178598e51a26abbc958b8a2e48825c90bc22e641de3d31e18aaf55f3258ba93b: not found"
    panic: rpc error: code = Unknown desc = content digest sha256:178598e51a26abbc958b8a2e48825c90bc22e641de3d31e18aaf55f3258ba93b: not found

It seems safe to just ignore these errors, since `NotFound` is what we wanted
after delete anyway.

This allows support for e.g. `llb.Image("docker.io/docker/whalesay:latest")`

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-02-26 17:08:52 +00:00
Tõnis Tiigi 96ad291c82
Merge pull request #295 from AkihiroSuda/reusable-docker-exporter-
exporter/oci: split reusable dockerexporter
2018-02-26 08:20:11 -08:00
Tõnis Tiigi 9ee3800a08
Merge pull request #290 from tonistiigi/session-prefix
session: clear prefix on lookup
2018-02-26 07:47:58 -08:00
Tõnis Tiigi aa58489e10
Merge pull request #294 from AkihiroSuda/runc-split-snapshotter
worker/oci: allow specifying arbitrary snapshotter factory
2018-02-26 07:33:06 -08:00
Akihiro Suda de73349f01 worker/oci: allow specifying arbitrary snapshotter factory
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-02-26 18:15:10 +09:00
Akihiro Suda e008553762 exporter/oci: split reusable dockerexporter
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-02-26 18:02:58 +09:00
Akihiro Suda d99b929a64
Merge pull request #292 from tonistiigi/cache-storage
solver: separate cache metadata storage interface
2018-02-26 14:00:46 +09:00
Tonis Tiigi ff7d75def0 solver: separate cache metadata storage interface
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-23 17:44:56 -08:00
Tonis Tiigi b7424f41fd session: clear prefix on lookup
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-23 10:52:55 -08:00