Commit Graph

207 Commits (19612b901faa957766805bda70a7ab0956273874)

Author SHA1 Message Date
Tibor Vass b4f7a7670d Move builder files to builder/dockerfile
Signed-off-by: Tibor Vass <tibor@docker.com>

rewritten from github.com/moby/moby f41230b93a77ca7f4d84718f39a4847b6117f694
2018-06-01 20:10:12 -07:00
Tonis Tiigi 6b25a56906 dockerfile: support building from a Dockerfile URL
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-30 13:20:27 -07:00
Tonis Tiigi 1067c24df2 git: support all git transports
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-29 10:56:08 -07:00
Tibor Vass 9adfee1475 dockerfile: use updated tonistiigi/copy:v0.1.1 image
https://github.com/tonistiigi/copy/pull/3

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-05-24 20:57:45 +00:00
Tõnis Tiigi 6a103ab0ac
Merge pull request #403 from tonistiigi/modtime
Fixes for cache invalidation on modtime change
2018-05-24 13:51:50 -07:00
Tonis Tiigi 5fbaff4f66 dockerfile: add test for cmd override and shell
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-23 13:27:35 -07:00
Tonis Tiigi 1fad598adf dockerfile: add withShell
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-23 12:19:10 -07:00
Tonis Tiigi 50386f3772 dockerfile: reset cmd from parent on entrypoint
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-23 12:18:14 -07:00
Tonis Tiigi eefc49ff72 dockerfile: detect empty images early
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-22 16:54:04 -07:00
Tonis Tiigi 8084e8cde9 llbsolver: allow empty returns for opts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-22 15:46:32 -07:00
Tonis Tiigi 53738b8211 dockerfile: allow ignorecache on copy
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-22 10:50:36 -07:00
Akihiro Suda 779d8fa5b2
Merge pull request #400 from tonistiigi/proxy-args
Add proxy args support
2018-05-22 16:04:12 +09:00
Tonis Tiigi a9dea8b235 dockerfile: add global arg test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-21 17:10:21 -07:00
Tonis Tiigi 286cf44db3 dockerfile: add default build-arg support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-21 16:08:12 -07:00
Akihiro Suda 199eb01177 dockerfile: fix `ADD https://.../foo.tar.gz /local/foo.tar.gz`
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-05-21 15:15:27 +09:00
Tonis Tiigi 1e096c4ce9 dockerfile: allow builds from http context
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-16 23:03:52 -07:00
Akihiro Suda 10003f2c2d
Merge pull request #384 from tonistiigi/syntax-directive
dockerfile: add syntax directive for introducing new features
2018-05-17 11:16:47 +09:00
Akihiro Suda 80f113196d
Merge pull request #390 from tiborvass/fix-nil-labels-panic
dockerfile: ensure Labels map is not accessed when nil
2018-05-17 11:15:10 +09:00
Akihiro Suda c4bffb7e06
Merge pull request #382 from moby/llbsolver-next
Upgrade to llbsolver-next
2018-05-17 11:14:23 +09:00
Tibor Vass a2d2551dcd dockerfile: ensure Labels map is not accessed when nil
Signed-off-by: Tibor Vass <tibor@docker.com>
2018-05-17 01:30:19 +00:00
Tibor Vass ae37b85b24 dockerfile: fix hanging when encountering invalid dockerignore pattern
This commit fixes the case when "!" is provided alone as a dockerignore
pattern resulting in buildkit hanging. An integration test is added to
guard the bug.

The bug was due to incorrect error propagation in the fsutil package.
Thus this commit vendors a newer, fixed version of fsutil.

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-05-16 23:26:43 +00:00
Tonis Tiigi 460fb33da7 dockerfile: add syntax directive for introducing new features
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-15 10:00:48 -07:00
Tonis Tiigi 71f5e6dd21 solver: rename solver-next to solver
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi a55d206cdb dockerfile: add no-cache support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi a7bc9b9fd2 cache: maintain creation time with remote cache
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi cea5d60dfa dockerfile: add cache import/export test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi 644540e0f1 gateway: update gateway to cache import refs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi 966d3f7b32 dockerfile: add cache-from support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi 11ada4c27b frontend: enable importref for frontends
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi 3ca07f1467 api: allow multiple cache import refs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi b6c782d53e control: switch to use solver-next
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Akihiro Suda 72c08b5cb9 update containerd (binary: v1.1.0, library: May 11, 2018)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-05-11 15:46:15 +09:00
Tonis Tiigi b80f45955a solver: allow exporters to set data to solve results
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-07 20:52:11 -07:00
Tonis Tiigi d51a921ab4 dockerfile: add test making sure cache generates same id
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-04 15:51:26 -07:00
Tonis Tiigi b8b55a8c22 image: export reproducible timestamps
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-04 15:51:19 -07:00
Tonis Tiigi f9d1e4c32c dockerfile: add test for labels
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-03 16:24:26 -07:00
Tonis Tiigi e24e194078 remove net/context references
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-30 18:10:54 -07:00
Tonis Tiigi e5e8755dff dockerfile: add tests for case insensitive targets
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-27 10:08:27 -07:00
Tonis Tiigi 864adf02dd dockerfile: fix context path detection for var substitution
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-23 11:37:53 -07:00
Tonis Tiigi 8b85812dda dockerfile: allow skipping resolving manifest
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-19 21:39:58 -07:00
Tonis Tiigi 0b203b21df dockerfile: copy only files that are used
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-13 21:13:29 -07:00
Tõnis Tiigi 89f8bc1c02
Merge pull request #329 from AkihiroSuda/containerd-20180403
update containerd (binary: v1.0.3, library: Apr 3, 2018)
2018-04-04 10:05:42 -07:00
Akihiro Suda 126f91dea7 update containerd (binary: v1.0.3, library: Apr 3, 2018)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-04-04 17:22:20 +09:00
Akihiro Suda aabfa3b84a
Merge pull request #324 from ijc/readonly-rootfs-execop
Make llb.ReadonlyRootFS usable with common container images
2018-04-04 11:43:44 +09:00
Ian Campbell d9b64588cb Regenerate protobuf go file with gogoproto v0.5
This matches the version in vendor.conf, the previous version was `v0.5~25`
according to `git describe`.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 13:45:50 +01:00
Ian Campbell ddae5a6ea4 Make llb.ReadonlyRootFS usable with common container images
e.g. with busybox image:

    OCI runtime create failed: container_linux.go:348:
    starting container process caused "process_linux.go:402:
    container init caused \"rootfs_linux.go:58:
    mounting \\\"proc\\\" to rootfs \\\"/.../rootfs\\\" at \\\"/proc\\\"
    caused \\\"mkdir /.../rootfs/proc: read-only file system\\\"\"": unknown

This is because we were setting the underlying snapshot readonly so the various
mountpoints (here /proc) cannot be created. This would not be necessary if
those mountpoints were present in images but they typically are not.

The right way to get around this (used e.g. by `ctr`) is to use a writeable
snapshot but to set root readonly in the OCI spec. In this configuration the
rootfs is writeable when mounts are processed but is then made readonly by the
runtime (runc) just before entering the user specified binary within the
container.

This involved a surprising amount of plumbing.

Use this new found ability in the dockerfile converter's `dispatchCopy`
function.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 11:04:07 +01:00
Tõnis Tiigi 44b843d50d
Merge pull request #323 from ijc/protobuf-regen
Validation and (controlled) generation for go generate'd files.
2018-03-28 10:12:14 -07:00
Ian Campbell 17069fe7e4 Validation and (controlled) generation for go generate'd files.
Modelled after the vendor support provide a validator and an updator for files
produced by `go generate` (which today just means `*.pb.go`).

Main difference from the vendor support is that we are no longer simply nuking
and replacing a single directory, so I ended up hardcoding `*.pb.go` in a bunch
of places which I don't like but cannot see a way around which doesn't risk
nuking people's other local changes.

The generated files are placed in an unpacked form in a `FROM scratch`
container for update. Use a subdirectory and `tar --strip-components` (portable
to MacOS and Linux according to `tar(1)`) since trying to do a `docker export`
of just the root ends up adding `.dockerenv`, `sys`, `proc` and `dev` to the
source tree.

The validate container is not `FROM scratch` because we want `cat`.

The run in `frontend/gateway/pb/generate.go` was missing an include so fix
that.

The versions of `protoc` and the gogo plugins were chosen to regenerate the
existing code as closely as possible. The updates to `*.pg.go` here are all the
result of regenerating with go1.9 which fixed
https://github.com/golang/go/issues/17663 and replaced an invalid timestamp in
the gzip header of the data encoded in `fileDescriptor*`, and adopted a new
standard for marking generated files.

Finally, I noticed that my `docker run`s were missing an `--rm` which I
inherited from `validate-vendor`, so fix all those.

Closes: #322

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-28 10:52:16 +01:00
Akihiro Suda 9ef8233da1 client: support passing io.WriteCloser via SolveOpt for FSSyncTargetFile
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-03-28 15:48:54 +09:00
Tonis Tiigi 12fd7edc24 gateway: typo fix
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-08 10:13:26 -08:00
Tonis Tiigi cd110f6b10 dockerfile: fix copy from implicit stage
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-21 18:17:30 -08:00
Tonis Tiigi 7906fb3085 vendor: update containerd to v1.0.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-13 16:00:09 -08:00
Tonis Tiigi ad6ab8aba2 dockerfile: new tests for copy
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-05 14:33:48 -05:00
Akihiro Suda 994c1a63af
Merge pull request #275 from vdemeester/add-linter-deadcode
Add deadcode to the linter
2018-02-02 11:13:29 +09:00
Vincent Demeester 79e346f59a
Add deadcode to the linter
it tends to help keeping uneeded code out O:)

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-02-01 17:59:04 -08:00
Vincent Demeester 2f162c1138
Remove shell parser for dockerfile2llb, using moby upstream package
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-02-01 17:46:51 -08:00
Vincent Demeester 08373f2381
Clean some small dependencies on docker/docker package
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-02-01 16:05:24 -08:00
Tonis Tiigi 6e40e83d35 Remove net/context dependencies
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-17 09:11:22 -08:00
Tonis Tiigi 29b72a3912 solver: add some trace points
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-08 18:03:12 -08:00
Tonis Tiigi 6cb4256e81 dockerfile: fix copy with wildcards and add tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-26 15:41:09 -08:00
Tonis Tiigi 3407de2e04 gateway: allow skipping tag in source parameter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-20 22:51:33 -08:00
Tonis Tiigi 7f513a4fd4 dockerfile: add copy —-chown support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-19 20:54:26 -08:00
Akihiro Suda 2a33b2e822 *: update build tags (`containerd` -> `no_oci_worker`)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-18 15:47:56 +09:00
Tonis Tiigi 367752bf1c dockerfile: add test for building from git repo
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-17 21:09:20 -08:00
Tonis Tiigi 2d3f36d359 source: add sharedkeyhint to local source
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-15 15:13:15 -08:00
Tonis Tiigi 08e1c2990c dockerfile: add dockerignore support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-15 15:03:35 -08:00
Tonis Tiigi 5d457719be dockerfile: fix wrong context passed to build
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-13 21:17:50 -08:00
Tonis Tiigi 46e816965b gateway: fix returning nil references
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-13 17:29:03 -08:00
Tonis Tiigi fc5d833cc1 gateway: fix unreleased references
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-13 16:51:31 -08:00
Tonis Tiigi 98e91f8b70 frontend: make dockerfile.v0 and external frontend use same code
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-13 16:51:25 -08:00
Akihiro Suda 571e14e003
Merge pull request #203 from tonistiigi/user
Add support for setting user
2017-12-13 13:06:07 +09:00
Tonis Tiigi e4f98e252d dockerfile: avoid pulling config for unreachable stages
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-12 15:02:19 -08:00
Tonis Tiigi 6626255a71 dockerfile: add support for user
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-12 10:28:37 -08:00
Tõnis Tiigi 60ea0babb5
Merge pull request #176 from AkihiroSuda/multiworker
multi-worker daemon
2017-12-12 10:09:03 -08:00
Akihiro Suda c3aa849014 multi-worker daemon
- [X] put multiples workers in a single binary ("-tags containerd standalone")
- [X] add worker selector to LLB vertex metadata
- [X] s/worker/executor/g
- [X] introduce the new "worker" concept https://github.com/moby/buildkit/pull/176#discussion_r153693928
- [X] fix up CLI
- [X] fix up tests
- allow using multiples workers (requires inter-vertex cache copier, HUGE!) --> will be separate PR

Implementation notes:
- "Workers" are renamed to "executors" now
- The new "worker" instance holds an "executor" instance and its
related stuffs such as the snapshotter
- The default worker is "runc-overlay"

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-12 15:17:58 +09:00
Tonis Tiigi 72afc856c8 dockerfile: add support for EXPOSE args
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-11 16:35:31 -08:00
Tonis Tiigi 6963b77c66 dockerfile: add tests for history records
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-08 20:00:36 -08:00
Tonis Tiigi 1f93a77f78 dockerfile: fill history array
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-08 20:00:36 -08:00
Tonis Tiigi e4f6a5e752 set more tests to parallel
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-08 18:19:08 -08:00
Tonis Tiigi 438c850a16 dockerfile: add original command metadata to llb
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-07 16:13:50 -08:00
Tonis Tiigi 68165c07f4 exporter: allow exporting config with no snapshots
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-04 23:18:09 -08:00
Tonis Tiigi b3568e7dcb dockerfile: return valid empty image config for scratch
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-04 23:16:23 -08:00
Tonis Tiigi 1b9c907ee4 dockerfile: add archive ADD support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-04 22:48:02 -08:00
Tonis Tiigi e0af9d1218 source: add more properties to http source
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-04 12:00:46 -08:00
Tonis Tiigi fc2f1687f9 dockerfile: add adding from URL support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-02 21:45:41 -08:00
Akihiro Suda 9653c6079c
Merge pull request #183 from tonistiigi/dockerfile-tests
frontend: add some dockerfile integration tests
2017-12-02 16:51:13 +09:00
Tonis Tiigi 2729373a03 reduce dependencies of llb package
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-30 17:29:57 -08:00
Tonis Tiigi bcc98bf29d frontend: add some dockerfile integration tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-30 14:57:18 -08:00
Tonis Tiigi b158114a6f dockerfile: fix copy to non-existing working dir
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-06 16:28:30 -08:00
Tonis Tiigi a6b1e4e399 dockerfile: allow building dockerfiles from git
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-06 16:27:20 -08:00
Tonis Tiigi 473346d1a5 update to go1.9
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-01 18:22:56 -07:00
Tonis Tiigi 56e1f71e6b cache: allow pushing exported cache
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-13 15:37:55 -07:00
Tonis Tiigi 23a114a977 solver: simplify solver public api
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-05 10:05:38 -07:00
Tonis Tiigi 96d39a8ba8 dockerfile: move image frontend to this repo until api stabilizes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-03 21:14:13 -07:00
Tonis Tiigi 80bc5f6097 fix exporter attributes interface returns
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-03 21:14:13 -07:00
Tonis Tiigi 2809d01cf6 frontend: add gateway frontend / image frontend support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-03 21:14:03 -07:00
Akihiro Suda f89ffc2ab4 llb: per-vertex metadata (e.g. IgnoreCache)
cache invalidation itself it not implemented yet

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-10-03 05:36:24 +00:00
Tonis Tiigi 05c39612d8 dockerfile: resolve to digest ref in frontend
Less hacky than keeping the cached resolver.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-22 15:49:37 -07:00
Tonis Tiigi 0995c119bf dockerfile: implement onbuild support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-22 13:12:57 -07:00
Tonis Tiigi d258bcab7b dockerfile: clean up converter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-22 10:49:19 -07:00
Tonis Tiigi fd0f7ee2a8 frontend: fix dockerfile frontend resolver
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-15 18:09:28 -07:00
Tonis Tiigi 5a38d573fa frontend: add support for build args for dockerfile
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-15 18:09:28 -07:00
Tonis Tiigi 4c22f6c25a frontend: share attributes to exporter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-15 18:09:28 -07:00
Tonis Tiigi b36df4738a frontend: add more commands to dockerfile
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-15 18:09:28 -07:00
Tonis Tiigi 6d335c3c86 imagesource: share config resolver with frontend
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-05 13:11:27 -07:00
Tonis Tiigi 4040029810 dockerfile: improve copy implementation
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-01 19:27:54 -07:00
Tonis Tiigi 2d290db465 frontend: base of dockerfile frontend
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-31 23:27:25 -07:00