Commit Graph

727 Commits (13665367df7421a8900f69a5ce641fbab9fe21b7)

Author SHA1 Message Date
Tonis Tiigi 864adf02dd dockerfile: fix context path detection for var substitution
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-23 11:37:53 -07:00
Tonis Tiigi 8b85812dda dockerfile: allow skipping resolving manifest
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-19 21:39:58 -07:00
Tonis Tiigi 0b203b21df dockerfile: copy only files that are used
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-13 21:13:29 -07:00
Tõnis Tiigi 89f8bc1c02
Merge pull request #329 from AkihiroSuda/containerd-20180403
update containerd (binary: v1.0.3, library: Apr 3, 2018)
2018-04-04 10:05:42 -07:00
Akihiro Suda 126f91dea7 update containerd (binary: v1.0.3, library: Apr 3, 2018)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-04-04 17:22:20 +09:00
Akihiro Suda aabfa3b84a
Merge pull request #324 from ijc/readonly-rootfs-execop
Make llb.ReadonlyRootFS usable with common container images
2018-04-04 11:43:44 +09:00
Ian Campbell d9b64588cb Regenerate protobuf go file with gogoproto v0.5
This matches the version in vendor.conf, the previous version was `v0.5~25`
according to `git describe`.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 13:45:50 +01:00
Ian Campbell ddae5a6ea4 Make llb.ReadonlyRootFS usable with common container images
e.g. with busybox image:

    OCI runtime create failed: container_linux.go:348:
    starting container process caused "process_linux.go:402:
    container init caused \"rootfs_linux.go:58:
    mounting \\\"proc\\\" to rootfs \\\"/.../rootfs\\\" at \\\"/proc\\\"
    caused \\\"mkdir /.../rootfs/proc: read-only file system\\\"\"": unknown

This is because we were setting the underlying snapshot readonly so the various
mountpoints (here /proc) cannot be created. This would not be necessary if
those mountpoints were present in images but they typically are not.

The right way to get around this (used e.g. by `ctr`) is to use a writeable
snapshot but to set root readonly in the OCI spec. In this configuration the
rootfs is writeable when mounts are processed but is then made readonly by the
runtime (runc) just before entering the user specified binary within the
container.

This involved a surprising amount of plumbing.

Use this new found ability in the dockerfile converter's `dispatchCopy`
function.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 11:04:07 +01:00
Tõnis Tiigi 44b843d50d
Merge pull request #323 from ijc/protobuf-regen
Validation and (controlled) generation for go generate'd files.
2018-03-28 10:12:14 -07:00
Ian Campbell 17069fe7e4 Validation and (controlled) generation for go generate'd files.
Modelled after the vendor support provide a validator and an updator for files
produced by `go generate` (which today just means `*.pb.go`).

Main difference from the vendor support is that we are no longer simply nuking
and replacing a single directory, so I ended up hardcoding `*.pb.go` in a bunch
of places which I don't like but cannot see a way around which doesn't risk
nuking people's other local changes.

The generated files are placed in an unpacked form in a `FROM scratch`
container for update. Use a subdirectory and `tar --strip-components` (portable
to MacOS and Linux according to `tar(1)`) since trying to do a `docker export`
of just the root ends up adding `.dockerenv`, `sys`, `proc` and `dev` to the
source tree.

The validate container is not `FROM scratch` because we want `cat`.

The run in `frontend/gateway/pb/generate.go` was missing an include so fix
that.

The versions of `protoc` and the gogo plugins were chosen to regenerate the
existing code as closely as possible. The updates to `*.pg.go` here are all the
result of regenerating with go1.9 which fixed
https://github.com/golang/go/issues/17663 and replaced an invalid timestamp in
the gzip header of the data encoded in `fileDescriptor*`, and adopted a new
standard for marking generated files.

Finally, I noticed that my `docker run`s were missing an `--rm` which I
inherited from `validate-vendor`, so fix all those.

Closes: #322

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-28 10:52:16 +01:00
Akihiro Suda 9ef8233da1 client: support passing io.WriteCloser via SolveOpt for FSSyncTargetFile
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-03-28 15:48:54 +09:00
Tonis Tiigi 12fd7edc24 gateway: typo fix
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-08 10:13:26 -08:00
Tonis Tiigi cd110f6b10 dockerfile: fix copy from implicit stage
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-21 18:17:30 -08:00
Tonis Tiigi 7906fb3085 vendor: update containerd to v1.0.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-13 16:00:09 -08:00
Daniel Nephin f6e9a96636 Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-05 16:51:57 -05:00
Tonis Tiigi ad6ab8aba2 dockerfile: new tests for copy
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-05 14:33:48 -05:00
Akihiro Suda 994c1a63af
Merge pull request #275 from vdemeester/add-linter-deadcode
Add deadcode to the linter
2018-02-02 11:13:29 +09:00
Vincent Demeester 79e346f59a
Add deadcode to the linter
it tends to help keeping uneeded code out O:)

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-02-01 17:59:04 -08:00
Vincent Demeester 2f162c1138
Remove shell parser for dockerfile2llb, using moby upstream package
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-02-01 17:46:51 -08:00
Vincent Demeester 08373f2381
Clean some small dependencies on docker/docker package
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-02-01 16:05:24 -08:00
Tonis Tiigi 6e40e83d35 Remove net/context dependencies
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-17 09:11:22 -08:00
Tonis Tiigi 29b72a3912 solver: add some trace points
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-08 18:03:12 -08:00
Tonis Tiigi 6cb4256e81 dockerfile: fix copy with wildcards and add tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-26 15:41:09 -08:00
Tonis Tiigi 3407de2e04 gateway: allow skipping tag in source parameter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-20 22:51:33 -08:00
Tonis Tiigi 7f513a4fd4 dockerfile: add copy —-chown support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-19 20:54:26 -08:00
Akihiro Suda 2a33b2e822 *: update build tags (`containerd` -> `no_oci_worker`)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-18 15:47:56 +09:00
Tonis Tiigi 367752bf1c dockerfile: add test for building from git repo
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-17 21:09:20 -08:00
Tonis Tiigi 2d3f36d359 source: add sharedkeyhint to local source
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-15 15:13:15 -08:00
Tonis Tiigi 08e1c2990c dockerfile: add dockerignore support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-15 15:03:35 -08:00
Tonis Tiigi 5d457719be dockerfile: fix wrong context passed to build
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-13 21:17:50 -08:00
Tonis Tiigi 46e816965b gateway: fix returning nil references
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-13 17:29:03 -08:00
Tonis Tiigi fc5d833cc1 gateway: fix unreleased references
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-13 16:51:31 -08:00
Tonis Tiigi 98e91f8b70 frontend: make dockerfile.v0 and external frontend use same code
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-13 16:51:25 -08:00
Akihiro Suda 571e14e003
Merge pull request #203 from tonistiigi/user
Add support for setting user
2017-12-13 13:06:07 +09:00
Tonis Tiigi e4f98e252d dockerfile: avoid pulling config for unreachable stages
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-12 15:02:19 -08:00
Tonis Tiigi 6626255a71 dockerfile: add support for user
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-12 10:28:37 -08:00
Tõnis Tiigi 60ea0babb5
Merge pull request #176 from AkihiroSuda/multiworker
multi-worker daemon
2017-12-12 10:09:03 -08:00
Akihiro Suda c3aa849014 multi-worker daemon
- [X] put multiples workers in a single binary ("-tags containerd standalone")
- [X] add worker selector to LLB vertex metadata
- [X] s/worker/executor/g
- [X] introduce the new "worker" concept https://github.com/moby/buildkit/pull/176#discussion_r153693928
- [X] fix up CLI
- [X] fix up tests
- allow using multiples workers (requires inter-vertex cache copier, HUGE!) --> will be separate PR

Implementation notes:
- "Workers" are renamed to "executors" now
- The new "worker" instance holds an "executor" instance and its
related stuffs such as the snapshotter
- The default worker is "runc-overlay"

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-12 15:17:58 +09:00
Tonis Tiigi 72afc856c8 dockerfile: add support for EXPOSE args
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-11 16:35:31 -08:00
Tonis Tiigi 6963b77c66 dockerfile: add tests for history records
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-08 20:00:36 -08:00
Tonis Tiigi 1f93a77f78 dockerfile: fill history array
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-08 20:00:36 -08:00
Tonis Tiigi e4f6a5e752 set more tests to parallel
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-08 18:19:08 -08:00
Tonis Tiigi 438c850a16 dockerfile: add original command metadata to llb
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-07 16:13:50 -08:00
Tonis Tiigi 68165c07f4 exporter: allow exporting config with no snapshots
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-04 23:18:09 -08:00
Tonis Tiigi b3568e7dcb dockerfile: return valid empty image config for scratch
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-04 23:16:23 -08:00
Tonis Tiigi 1b9c907ee4 dockerfile: add archive ADD support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-04 22:48:02 -08:00
Tonis Tiigi e0af9d1218 source: add more properties to http source
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-04 12:00:46 -08:00
Tonis Tiigi fc2f1687f9 dockerfile: add adding from URL support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-02 21:45:41 -08:00
Akihiro Suda 9653c6079c
Merge pull request #183 from tonistiigi/dockerfile-tests
frontend: add some dockerfile integration tests
2017-12-02 16:51:13 +09:00
Tonis Tiigi 2729373a03 reduce dependencies of llb package
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-30 17:29:57 -08:00
Tonis Tiigi bcc98bf29d frontend: add some dockerfile integration tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-30 14:57:18 -08:00
Tonis Tiigi b158114a6f dockerfile: fix copy to non-existing working dir
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-06 16:28:30 -08:00
Tonis Tiigi a6b1e4e399 dockerfile: allow building dockerfiles from git
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-06 16:27:20 -08:00
Tonis Tiigi 473346d1a5 update to go1.9
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-01 18:22:56 -07:00
Tonis Tiigi 56e1f71e6b cache: allow pushing exported cache
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-13 15:37:55 -07:00
Tonis Tiigi 23a114a977 solver: simplify solver public api
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-05 10:05:38 -07:00
Tonis Tiigi 96d39a8ba8 dockerfile: move image frontend to this repo until api stabilizes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-03 21:14:13 -07:00
Tonis Tiigi 80bc5f6097 fix exporter attributes interface returns
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-03 21:14:13 -07:00
Tonis Tiigi 2809d01cf6 frontend: add gateway frontend / image frontend support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-03 21:14:03 -07:00
Akihiro Suda f89ffc2ab4 llb: per-vertex metadata (e.g. IgnoreCache)
cache invalidation itself it not implemented yet

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-10-03 05:36:24 +00:00
Tonis Tiigi 05c39612d8 dockerfile: resolve to digest ref in frontend
Less hacky than keeping the cached resolver.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-22 15:49:37 -07:00
Tonis Tiigi 0995c119bf dockerfile: implement onbuild support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-22 13:12:57 -07:00
Tonis Tiigi d258bcab7b dockerfile: clean up converter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-22 10:49:19 -07:00
Tonis Tiigi fd0f7ee2a8 frontend: fix dockerfile frontend resolver
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-15 18:09:28 -07:00
Tonis Tiigi 5a38d573fa frontend: add support for build args for dockerfile
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-15 18:09:28 -07:00
Tonis Tiigi 4c22f6c25a frontend: share attributes to exporter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-15 18:09:28 -07:00
Tonis Tiigi b36df4738a frontend: add more commands to dockerfile
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-15 18:09:28 -07:00
Tonis Tiigi 6d335c3c86 imagesource: share config resolver with frontend
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-05 13:11:27 -07:00
Tonis Tiigi 4040029810 dockerfile: improve copy implementation
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-01 19:27:54 -07:00
Tonis Tiigi 2d290db465 frontend: base of dockerfile frontend
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-31 23:27:25 -07:00
Simon Ferquel 1155aeccf1 Fix behavior of absolute paths in .dockerignore
According to documentation (https://docs.docker.com/engine/reference/builder/#dockerignore-file), absolute paths like `/foo/bar` should have the same effect as `foo/bar`. This is not the case today.

This fix normalize paths when reading the .dockerignore file by removing
leading slashes.

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
2017-03-31 10:43:59 +02:00
fate-grand-order 54ee06727e use t.Fatal() to output the err message where the values used for formatting
text does not appear to contain a placeholder

Signed-off-by: Helen Xie <chenjg@harmonycloud.cn>
2017-02-23 10:16:22 +08:00
allencloud c450963994 add defer file.Close to avoid potential fd leak
Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-08-10 08:36:09 +08:00
Yong Tang ddd1b33b56 Skip UTF-8 BOM bytes from Dockerignore if exist
This fix tries to address issues related to #23221 where Dockerignore
may consists of UTF-8 BOM. This likely happens when Notepad
tries to save a file as UTF-8 in Windows.

This fix skips the UTF-8 BOM bytes from the beginning of the
Dockerignore if exists.

Additional tests has been added to cover the changes in this fix.

This fix is related to #23221 (UTF-8 BOM in Dockerfile).

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2016-06-03 07:26:36 -07:00
Yong Tang 3666790305 Add support for comment in .dockerignore
This fix tries to address the issue raised in #20083 where
comment is not supported in `.dockerignore`.

This fix updated the processing of `.dockerignore` so that any
lines starting with `#` are ignored, which is similiar to the
behavior of `.gitignore`.

Related documentation has been updated.

Additional tests have been added to cover the changes.

This fix fixes #20083.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2016-06-02 19:06:52 -07:00
Anusha Ragunathan 0c2e049180 Fix ReadAll to run on Windows.
filepath.Clean converts filenames to filenames with native path
separators. Use ToSlash to normalize.

Signed-off-by: Anusha Ragunathan <anusha@docker.com>
2016-02-04 14:01:17 -08:00
Tibor Vass e09818357f utils: move dockerignore function to builder/dockerignore
Signed-off-by: Tibor Vass <tibor@docker.com>
2015-12-14 14:59:52 +01:00