JohnHammond
/
buildkit
mirror of https://github.com/JohnHammond/buildkit.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
955 Commits
7 Branches
72 Tags
27 MiB
Commit Graph

194 Commits (eb4484544126c4118f4c0fe906427b58d021bf1c)

Author SHA1 Message Date
Tonis Tiigi 286cf44db3 dockerfile: add default build-arg support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-21 16:08:12 -07:00
Akihiro Suda 199eb01177 dockerfile: fix `ADD https://.../foo.tar.gz /local/foo.tar.gz` 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-05-21 15:15:27 +09:00
Tonis Tiigi 1e096c4ce9 dockerfile: allow builds from http context 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-16 23:03:52 -07:00
Akihiro Suda 10003f2c2d
Merge pull request #384 from tonistiigi/syntax-directive 
dockerfile: add syntax directive for introducing new features
 2018-05-17 11:16:47 +09:00
Akihiro Suda 80f113196d
Merge pull request #390 from tiborvass/fix-nil-labels-panic 
dockerfile: ensure Labels map is not accessed when nil
 2018-05-17 11:15:10 +09:00
Akihiro Suda c4bffb7e06
Merge pull request #382 from moby/llbsolver-next 
Upgrade to llbsolver-next
 2018-05-17 11:14:23 +09:00
Tibor Vass a2d2551dcd dockerfile: ensure Labels map is not accessed when nil 
Signed-off-by: Tibor Vass <tibor@docker.com>
 2018-05-17 01:30:19 +00:00
Tibor Vass ae37b85b24 dockerfile: fix hanging when encountering invalid dockerignore pattern 
This commit fixes the case when "!" is provided alone as a dockerignore
pattern resulting in buildkit hanging. An integration test is added to
guard the bug.

The bug was due to incorrect error propagation in the fsutil package.
Thus this commit vendors a newer, fixed version of fsutil.

Signed-off-by: Tibor Vass <tibor@docker.com>
 2018-05-16 23:26:43 +00:00
Tonis Tiigi 460fb33da7 dockerfile: add syntax directive for introducing new features 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-15 10:00:48 -07:00
Tonis Tiigi 71f5e6dd21 solver: rename solver-next to solver 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi a55d206cdb dockerfile: add no-cache support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi a7bc9b9fd2 cache: maintain creation time with remote cache 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi cea5d60dfa dockerfile: add cache import/export test 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi 644540e0f1 gateway: update gateway to cache import refs 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi 966d3f7b32 dockerfile: add cache-from support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi 11ada4c27b frontend: enable importref for frontends 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi 3ca07f1467 api: allow multiple cache import refs 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi b6c782d53e control: switch to use solver-next 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Akihiro Suda 72c08b5cb9 update containerd (binary: v1.1.0, library: May 11, 2018) 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-05-11 15:46:15 +09:00
Tonis Tiigi b80f45955a solver: allow exporters to set data to solve results 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-07 20:52:11 -07:00
Tonis Tiigi d51a921ab4 dockerfile: add test making sure cache generates same id 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-04 15:51:26 -07:00
Tonis Tiigi b8b55a8c22 image: export reproducible timestamps 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-04 15:51:19 -07:00
Tonis Tiigi f9d1e4c32c dockerfile: add test for labels 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-03 16:24:26 -07:00
Tonis Tiigi e24e194078 remove net/context references 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-04-30 18:10:54 -07:00
Tonis Tiigi e5e8755dff dockerfile: add tests for case insensitive targets 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-04-27 10:08:27 -07:00
Tonis Tiigi 864adf02dd dockerfile: fix context path detection for var substitution 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-04-23 11:37:53 -07:00
Tonis Tiigi 8b85812dda dockerfile: allow skipping resolving manifest 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-04-19 21:39:58 -07:00
Tonis Tiigi 0b203b21df dockerfile: copy only files that are used 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-04-13 21:13:29 -07:00
Tõnis Tiigi 89f8bc1c02
Merge pull request #329 from AkihiroSuda/containerd-20180403 
update containerd (binary: v1.0.3, library: Apr 3, 2018)
 2018-04-04 10:05:42 -07:00
Akihiro Suda 126f91dea7 update containerd (binary: v1.0.3, library: Apr 3, 2018) 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-04-04 17:22:20 +09:00
Akihiro Suda aabfa3b84a
Merge pull request #324 from ijc/readonly-rootfs-execop 
Make llb.ReadonlyRootFS usable with common container images
 2018-04-04 11:43:44 +09:00
Ian Campbell d9b64588cb Regenerate protobuf go file with gogoproto v0.5 
This matches the version in vendor.conf, the previous version was `v0.5~25`
according to `git describe`.

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-04-03 13:45:50 +01:00
Ian Campbell ddae5a6ea4 Make llb.ReadonlyRootFS usable with common container images 
e.g. with busybox image:

    OCI runtime create failed: container_linux.go:348:
    starting container process caused "process_linux.go:402:
    container init caused \"rootfs_linux.go:58:
    mounting \\\"proc\\\" to rootfs \\\"/.../rootfs\\\" at \\\"/proc\\\"
    caused \\\"mkdir /.../rootfs/proc: read-only file system\\\"\"": unknown

This is because we were setting the underlying snapshot readonly so the various
mountpoints (here /proc) cannot be created. This would not be necessary if
those mountpoints were present in images but they typically are not.

The right way to get around this (used e.g. by `ctr`) is to use a writeable
snapshot but to set root readonly in the OCI spec. In this configuration the
rootfs is writeable when mounts are processed but is then made readonly by the
runtime (runc) just before entering the user specified binary within the
container.

This involved a surprising amount of plumbing.

Use this new found ability in the dockerfile converter's `dispatchCopy`
function.

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-04-03 11:04:07 +01:00
Tõnis Tiigi 44b843d50d
Merge pull request #323 from ijc/protobuf-regen 
Validation and (controlled) generation for go generate'd files.
 2018-03-28 10:12:14 -07:00
Ian Campbell 17069fe7e4 Validation and (controlled) generation for go generate'd files. 
Modelled after the vendor support provide a validator and an updator for files
produced by `go generate` (which today just means `*.pb.go`).

Main difference from the vendor support is that we are no longer simply nuking
and replacing a single directory, so I ended up hardcoding `*.pb.go` in a bunch
of places which I don't like but cannot see a way around which doesn't risk
nuking people's other local changes.

The generated files are placed in an unpacked form in a `FROM scratch`
container for update. Use a subdirectory and `tar --strip-components` (portable
to MacOS and Linux according to `tar(1)`) since trying to do a `docker export`
of just the root ends up adding `.dockerenv`, `sys`, `proc` and `dev` to the
source tree.

The validate container is not `FROM scratch` because we want `cat`.

The run in `frontend/gateway/pb/generate.go` was missing an include so fix
that.

The versions of `protoc` and the gogo plugins were chosen to regenerate the
existing code as closely as possible. The updates to `*.pg.go` here are all the
result of regenerating with go1.9 which fixed
https://github.com/golang/go/issues/17663 and replaced an invalid timestamp in
the gzip header of the data encoded in `fileDescriptor*`, and adopted a new
standard for marking generated files.

Finally, I noticed that my `docker run`s were missing an `--rm` which I
inherited from `validate-vendor`, so fix all those.

Closes: #322

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-03-28 10:52:16 +01:00
Akihiro Suda 9ef8233da1 client: support passing io.WriteCloser via SolveOpt for FSSyncTargetFile 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-03-28 15:48:54 +09:00
Tonis Tiigi 12fd7edc24 gateway: typo fix 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-03-08 10:13:26 -08:00
Tonis Tiigi cd110f6b10 dockerfile: fix copy from implicit stage 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-02-21 18:17:30 -08:00
Tonis Tiigi 7906fb3085 vendor: update containerd to v1.0.2 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-02-13 16:00:09 -08:00
Tonis Tiigi ad6ab8aba2 dockerfile: new tests for copy 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-02-05 14:33:48 -05:00
Akihiro Suda 994c1a63af
Merge pull request #275 from vdemeester/add-linter-deadcode 
Add deadcode to the linter
 2018-02-02 11:13:29 +09:00
Vincent Demeester 79e346f59a
Add deadcode to the linter 
it tends to help keeping uneeded code out O:)

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
 2018-02-01 17:59:04 -08:00
Vincent Demeester 2f162c1138
Remove shell parser for dockerfile2llb, using moby upstream package 
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
 2018-02-01 17:46:51 -08:00
Vincent Demeester 08373f2381
Clean some small dependencies on docker/docker package 
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
 2018-02-01 16:05:24 -08:00
Tonis Tiigi 6e40e83d35 Remove net/context dependencies 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-01-17 09:11:22 -08:00
Tonis Tiigi 29b72a3912 solver: add some trace points 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-01-08 18:03:12 -08:00
Tonis Tiigi 6cb4256e81 dockerfile: fix copy with wildcards and add tests 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-26 15:41:09 -08:00
Tonis Tiigi 3407de2e04 gateway: allow skipping tag in source parameter 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-20 22:51:33 -08:00
Tonis Tiigi 7f513a4fd4 dockerfile: add copy —-chown support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-19 20:54:26 -08:00
Akihiro Suda 2a33b2e822 *: update build tags (`containerd` -> `no_oci_worker`) 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-18 15:47:56 +09:00
Tonis Tiigi 367752bf1c dockerfile: add test for building from git repo 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-17 21:09:20 -08:00
Tonis Tiigi 2d3f36d359 source: add sharedkeyhint to local source 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-15 15:13:15 -08:00
Tonis Tiigi 08e1c2990c dockerfile: add dockerignore support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-15 15:03:35 -08:00
Tonis Tiigi 5d457719be dockerfile: fix wrong context passed to build 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-13 21:17:50 -08:00
Tonis Tiigi 46e816965b gateway: fix returning nil references 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-13 17:29:03 -08:00
Tonis Tiigi fc5d833cc1 gateway: fix unreleased references 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-13 16:51:31 -08:00
Tonis Tiigi 98e91f8b70 frontend: make dockerfile.v0 and external frontend use same code 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-13 16:51:25 -08:00
Akihiro Suda 571e14e003
Merge pull request #203 from tonistiigi/user 
Add support for setting user
 2017-12-13 13:06:07 +09:00
Tonis Tiigi e4f98e252d dockerfile: avoid pulling config for unreachable stages 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-12 15:02:19 -08:00
Tonis Tiigi 6626255a71 dockerfile: add support for user 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-12 10:28:37 -08:00
Tõnis Tiigi 60ea0babb5
Merge pull request #176 from AkihiroSuda/multiworker 
multi-worker daemon
 2017-12-12 10:09:03 -08:00
Akihiro Suda c3aa849014 multi-worker daemon 
- [X] put multiples workers in a single binary ("-tags containerd standalone")
- [X] add worker selector to LLB vertex metadata
- [X] s/worker/executor/g
- [X] introduce the new "worker" concept https://github.com/moby/buildkit/pull/176#discussion_r153693928
- [X] fix up CLI
- [X] fix up tests
- allow using multiples workers (requires inter-vertex cache copier, HUGE!) --> will be separate PR

Implementation notes:
- "Workers" are renamed to "executors" now
- The new "worker" instance holds an "executor" instance and its
related stuffs such as the snapshotter
- The default worker is "runc-overlay"

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-12 15:17:58 +09:00
Tonis Tiigi 72afc856c8 dockerfile: add support for EXPOSE args 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-11 16:35:31 -08:00
Tonis Tiigi 6963b77c66 dockerfile: add tests for history records 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-08 20:00:36 -08:00
Tonis Tiigi 1f93a77f78 dockerfile: fill history array 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-08 20:00:36 -08:00
Tonis Tiigi e4f6a5e752 set more tests to parallel 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-08 18:19:08 -08:00
Tonis Tiigi 438c850a16 dockerfile: add original command metadata to llb 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-07 16:13:50 -08:00
Tonis Tiigi 68165c07f4 exporter: allow exporting config with no snapshots 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-04 23:18:09 -08:00
Tonis Tiigi b3568e7dcb dockerfile: return valid empty image config for scratch 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-04 23:16:23 -08:00
Tonis Tiigi 1b9c907ee4 dockerfile: add archive ADD support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-04 22:48:02 -08:00
Tonis Tiigi e0af9d1218 source: add more properties to http source 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-04 12:00:46 -08:00
Tonis Tiigi fc2f1687f9 dockerfile: add adding from URL support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-02 21:45:41 -08:00
Akihiro Suda 9653c6079c
Merge pull request #183 from tonistiigi/dockerfile-tests 
frontend: add some dockerfile integration tests
 2017-12-02 16:51:13 +09:00
Tonis Tiigi 2729373a03 reduce dependencies of llb package 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-11-30 17:29:57 -08:00
Tonis Tiigi bcc98bf29d frontend: add some dockerfile integration tests 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-11-30 14:57:18 -08:00
Tonis Tiigi b158114a6f dockerfile: fix copy to non-existing working dir 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-11-06 16:28:30 -08:00
Tonis Tiigi a6b1e4e399 dockerfile: allow building dockerfiles from git 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-11-06 16:27:20 -08:00
Tonis Tiigi 473346d1a5 update to go1.9 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-11-01 18:22:56 -07:00
Tonis Tiigi 56e1f71e6b cache: allow pushing exported cache 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-13 15:37:55 -07:00
Tonis Tiigi 23a114a977 solver: simplify solver public api 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-05 10:05:38 -07:00
Tonis Tiigi 96d39a8ba8 dockerfile: move image frontend to this repo until api stabilizes 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-03 21:14:13 -07:00
Tonis Tiigi 80bc5f6097 fix exporter attributes interface returns 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-03 21:14:13 -07:00
Tonis Tiigi 2809d01cf6 frontend: add gateway frontend / image frontend support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-03 21:14:03 -07:00
Akihiro Suda f89ffc2ab4 llb: per-vertex metadata (e.g. IgnoreCache) 
cache invalidation itself it not implemented yet

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-10-03 05:36:24 +00:00
Tonis Tiigi 05c39612d8 dockerfile: resolve to digest ref in frontend 
Less hacky than keeping the cached resolver.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-22 15:49:37 -07:00
Tonis Tiigi 0995c119bf dockerfile: implement onbuild support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-22 13:12:57 -07:00
Tonis Tiigi d258bcab7b dockerfile: clean up converter 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-22 10:49:19 -07:00
Tonis Tiigi fd0f7ee2a8 frontend: fix dockerfile frontend resolver 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-15 18:09:28 -07:00
Tonis Tiigi 5a38d573fa frontend: add support for build args for dockerfile 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-15 18:09:28 -07:00
Tonis Tiigi 4c22f6c25a frontend: share attributes to exporter 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-15 18:09:28 -07:00
Tonis Tiigi b36df4738a frontend: add more commands to dockerfile 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-15 18:09:28 -07:00
Tonis Tiigi 6d335c3c86 imagesource: share config resolver with frontend 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-05 13:11:27 -07:00
Tonis Tiigi 4040029810 dockerfile: improve copy implementation 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-01 19:27:54 -07:00
Tonis Tiigi 2d290db465 frontend: base of dockerfile frontend 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-08-31 23:27:25 -07:00