Commit Graph

958 Commits (3c973bacfe0c58f71877635866b6c37afa17d2aa)

Author SHA1 Message Date
Akihiro Suda aabfa3b84a
Merge pull request #324 from ijc/readonly-rootfs-execop
Make llb.ReadonlyRootFS usable with common container images
2018-04-04 11:43:44 +09:00
Tõnis Tiigi 2aa3554778
Merge pull request #330 from ijc/bump-protobuf
Consolidate gogoproto versions
2018-04-03 10:01:47 -07:00
Ian Campbell 8a51342bd5 Correct instructional message
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 13:45:50 +01:00
Ian Campbell ec18d4ead1 generate-files: auto sync gogo version to vendor.conf
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 13:45:50 +01:00
Ian Campbell d9b64588cb Regenerate protobuf go file with gogoproto v0.5
This matches the version in vendor.conf, the previous version was `v0.5~25`
according to `git describe`.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 13:45:50 +01:00
Ian Campbell 2e6adc027d Reduce code duplication with a closure.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 11:13:15 +01:00
Ian Campbell ddae5a6ea4 Make llb.ReadonlyRootFS usable with common container images
e.g. with busybox image:

    OCI runtime create failed: container_linux.go:348:
    starting container process caused "process_linux.go:402:
    container init caused \"rootfs_linux.go:58:
    mounting \\\"proc\\\" to rootfs \\\"/.../rootfs\\\" at \\\"/proc\\\"
    caused \\\"mkdir /.../rootfs/proc: read-only file system\\\"\"": unknown

This is because we were setting the underlying snapshot readonly so the various
mountpoints (here /proc) cannot be created. This would not be necessary if
those mountpoints were present in images but they typically are not.

The right way to get around this (used e.g. by `ctr`) is to use a writeable
snapshot but to set root readonly in the OCI spec. In this configuration the
rootfs is writeable when mounts are processed but is then made readonly by the
runtime (runc) just before entering the user specified binary within the
container.

This involved a surprising amount of plumbing.

Use this new found ability in the dockerfile converter's `dispatchCopy`
function.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 11:04:07 +01:00
Ian Campbell 4bae684e32 Invert tags on runc worker test so they run by default.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 11:01:46 +01:00
Tõnis Tiigi 914fce58ee
Merge pull request #328 from AkihiroSuda/disallow-empty-output
client: disallow empty output for exportation
2018-04-01 21:44:17 -07:00
Akihiro Suda 3dd4d12796 client: disallow empty output for exportation
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-04-02 13:22:44 +09:00
Tõnis Tiigi 44b843d50d
Merge pull request #323 from ijc/protobuf-regen
Validation and (controlled) generation for go generate'd files.
2018-03-28 10:12:14 -07:00
Ian Campbell 17069fe7e4 Validation and (controlled) generation for go generate'd files.
Modelled after the vendor support provide a validator and an updator for files
produced by `go generate` (which today just means `*.pb.go`).

Main difference from the vendor support is that we are no longer simply nuking
and replacing a single directory, so I ended up hardcoding `*.pb.go` in a bunch
of places which I don't like but cannot see a way around which doesn't risk
nuking people's other local changes.

The generated files are placed in an unpacked form in a `FROM scratch`
container for update. Use a subdirectory and `tar --strip-components` (portable
to MacOS and Linux according to `tar(1)`) since trying to do a `docker export`
of just the root ends up adding `.dockerenv`, `sys`, `proc` and `dev` to the
source tree.

The validate container is not `FROM scratch` because we want `cat`.

The run in `frontend/gateway/pb/generate.go` was missing an include so fix
that.

The versions of `protoc` and the gogo plugins were chosen to regenerate the
existing code as closely as possible. The updates to `*.pg.go` here are all the
result of regenerating with go1.9 which fixed
https://github.com/golang/go/issues/17663 and replaced an invalid timestamp in
the gzip header of the data encoded in `fileDescriptor*`, and adopted a new
standard for marking generated files.

Finally, I noticed that my `docker run`s were missing an `--rm` which I
inherited from `validate-vendor`, so fix all those.

Closes: #322

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-28 10:52:16 +01:00
Vincent Demeester 117be599b5
Merge pull request #326 from AkihiroSuda/bud-pipe
client: support passing io.WriteCloser via SolveOpt for FSSyncTargetFile
2018-03-28 09:43:49 +02:00
Akihiro Suda 9ef8233da1 client: support passing io.WriteCloser via SolveOpt for FSSyncTargetFile
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-03-28 15:48:54 +09:00
Tõnis Tiigi a0a7301ea0
Merge pull request #312 from tonistiigi/throttle-gc
worker: throttle calls to gc
2018-03-26 09:34:35 -07:00
Tõnis Tiigi 5b499d557b
Merge pull request #321 from jessfraz/seccomp
add containerd package for default seccomp profile
2018-03-23 10:14:09 -07:00
Jess Frazelle 4158ef3851
add seccomp to binary building
Signed-off-by: Jess Frazelle <acidburn@microsoft.com>
2018-03-23 12:59:15 -04:00
Akihiro Suda 1cf2daae92
Merge pull request #320 from tonistiigi/contentutil
util: add contentutil helpers
2018-03-23 10:09:15 +09:00
Tonis Tiigi 0702303f6c imagemetaresolver: switch to contentutil buffer
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-22 16:19:56 -07:00
Tonis Tiigi 05da9869bd util: add contentutil helpers
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-22 16:19:49 -07:00
Jess Frazelle 863736d818
add containerd package for default seccomp profile
Signed-off-by: Jess Frazelle <acidburn@microsoft.com>
2018-03-22 12:25:55 -04:00
Tõnis Tiigi 9f8547022b
Merge pull request #319 from ijc/check-mount-source
Check that mounts have a Source when generating the OCI spec
2018-03-22 08:53:28 -07:00
Ian Campbell ac6598d255 Check that mounts always have a target.
Otherwise the daemon panics when generating the OCI spec.

For belt and braces check in the ExecOp Run function but also when generating the spec.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-22 12:05:12 +00:00
Tõnis Tiigi a1af6d9037
Merge pull request #318 from jessfraz/all-changes
export cache
2018-03-21 10:10:18 -07:00
Jess Frazelle b0f34cb8cd
export cache
Signed-off-by: Jess Frazelle <acidburn@microsoft.com>
2018-03-21 12:34:05 -04:00
Tõnis Tiigi 3e90755493
Merge pull request #316 from ijc/readonly-rootfs
Convert ReadonlyRootFS to a RunOption
2018-03-20 10:17:57 -07:00
Ian Campbell 7be9ae6954 Convert ReadonlyRootFS to a RunOption
This seems to have missed out on an update/refactor at some point and was not
usable in its previous form without duplicating the `runOptionFunc`
scaffolding.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-20 16:58:05 +00:00
Akihiro Suda 67baec79bd
Merge pull request #311 from tonistiigi/optimize-marshal
client: speed up llb marshal
2018-03-20 12:21:56 +09:00
Akihiro Suda f635b18790
Merge pull request #314 from tonistiigi/cancel-fix
solver: don’t cancel child pipes from state change
2018-03-20 12:21:33 +09:00
Akihiro Suda b99388e9dd
Merge pull request #315 from tonistiigi/example-fix
examples: fix gobuild example
2018-03-20 12:21:05 +09:00
Tonis Tiigi 8fd2d5431b examples: fix gobuild example
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-19 11:52:52 -07:00
Tonis Tiigi 5717a4f49f solver: don’t cancel child pipes from state change
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-19 11:26:46 -07:00
Tonis Tiigi da633a8e6b worker: throttle calls to gc
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-19 10:40:31 -07:00
Tonis Tiigi 432c191e81 client: speed up llb marshal
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-19 10:25:58 -07:00
Akihiro Suda e37d938d94
Merge pull request #310 from tonistiigi/faster-load
solver: faster vertex loading
2018-03-19 14:38:20 +09:00
Akihiro Suda d8e533b27b
Merge pull request #313 from tonistiigi/cache-interface
solver: simplify cache storage backend
2018-03-19 14:36:07 +09:00
Akihiro Suda bc7501291c
Merge pull request #307 from tonistiigi/solver-query
solver-next: add support for cache tracking through query
2018-03-19 13:26:20 +09:00
Tonis Tiigi 4df3b2e000 solver: simplify cache backend interface
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-18 21:25:55 -07:00
Tonis Tiigi 5593bb9f48 solver: optimize cache storage interface
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-18 21:25:47 -07:00
Tonis Tiigi 0d536d40b9 solver: faster vertex loading
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-18 20:56:49 -07:00
Tonis Tiigi 513018806b solver: add support for cache tracking through query
Fix result releasing bugs.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-15 15:33:56 -07:00
Akihiro Suda 12198eea27
Merge pull request #306 from tonistiigi/contenthash-fix
contenthash: fix ignored files from path separator
2018-03-14 12:11:44 +09:00
Tonis Tiigi 8f8221d173 contenthash: fix ignored files from path separator
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-13 14:40:27 -07:00
Tõnis Tiigi b403f71a62
Merge pull request #304 from tonistiigi/typo-fix
gateway: typo fix
2018-03-08 10:29:46 -08:00
Tonis Tiigi 12fd7edc24 gateway: typo fix
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-08 10:13:26 -08:00
Tõnis Tiigi abe6627919
Merge pull request #303 from ijc/vndr-fixes
vendoring: try to catch more vendoring issues & fixup an existing one
2018-03-08 10:03:52 -08:00
Ian Campbell 86912f507e vendor.conf: Readd github.com/morikuni/aec
Accidentally removed in faf258dd2e ("vendor: add go-units").

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-08 14:46:28 +00:00
Ian Campbell 0d34cceeb6 vendoring: try to catch more vendoring issues.
At the moment vendor.conf is missing entries for `github.com/tonistiigi/llb-gobuild`
and `github.com/morikuni/aec` due in part because of a combination of
https://github.com/LK4D4/vndr/issues/62 and https://github.com/LK4D4/vndr/issues/63.

The issue vndr#63 (related to lack of `github.com/morikuni/aec`) can be worked
around by removing the vendor directory before rerunning `vndr`, so do so.

Due to vndr#62 the issue with `github.com/tonistiigi/llb-gobuild` cannot be
detected at the moment, but pass `-strict` to `vndr` in anticipation of a fix
there.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-08 14:46:28 +00:00
Tõnis Tiigi e8dbd3aeb4
Merge pull request #302 from ijc/improved-schema1-support
Improved schema1 support
2018-03-07 09:41:34 -08:00
Ian Campbell ae0293c4bf Add an integration test for pulling schema1 images
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-07 15:56:22 +00:00