Commit Graph

700 Commits (43a74261b32a97d64cd7a3948e612bc223e4de9a)

Author SHA1 Message Date
Akihiro Suda cd89a8ce3d go 1.12
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-03-17 04:38:23 +09:00
Tonis Tiigi 5effbff314 dockerfile: improve symlinks copy tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:58 -07:00
Tibor Vass 6fd30e7b43 dockerfile: add more tests to fix fileop
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-03-15 17:49:58 -07:00
Tonis Tiigi c6149da2eb fileop: review fixes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:58 -07:00
Tonis Tiigi f38d971b95 dockerfile: add matrix testing for non-fileop versions
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:58 -07:00
Tonis Tiigi 637bec7196 dockerfile: make fileop default
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:58 -07:00
Tonis Tiigi c57e5b22ea dockerfile: add regression test for wildcard cache
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:58 -07:00
Tonis Tiigi c4ef668f62 dockerfile: regression test for existing dest dir perms
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:58 -07:00
Tonis Tiigi 369f7482de dockerfile: regression test for wildcard unpack
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:58 -07:00
Tonis Tiigi 39ba2ede04 dockerfile: regression test for workdir creation
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:58 -07:00
Tonis Tiigi bb11c8c5c9 dockerfile: empty wildcard regression test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:58 -07:00
Tonis Tiigi 0d17ac323e fileop: updates with new fsutil copy pkg
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:58 -07:00
Tonis Tiigi 8a4674bab4 fileop: add dockerfile support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:50 -07:00
Akihiro Suda 5c9f7b8ff0 buildctl: new CLI ("Option C+")
See https://github.com/moby/buildkit/pull/807#issuecomment-468146089

Close #774

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-03-06 13:20:21 +09:00
Tonis Tiigi 632797cef0 vendor: udpate fsutil to b4281fa
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-05 17:34:01 -08:00
Tonis Tiigi 62697a93f7 dockerfile: allow custom context names
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-02-23 09:12:54 -08:00
Tibor Vass 175506ff7f hack: update scripts to be go mod compatible
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-01-31 22:44:47 +00:00
Tõnis Tiigi 7679d6fa7d
Merge pull request #795 from tonistiigi/fmt-fix
client: avoid string formatting on custom vertex names
2019-01-23 08:30:08 -08:00
Tonis Tiigi c0371c908f client: avoid string formatting on custom vertex names
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-01-22 15:35:39 -08:00
Tonis Tiigi 03a2f79561 gateway: allow frontends without network
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-01-21 19:38:07 -08:00
Tonis Tiigi db3db1033d dockerfile: ensure metadata commands have created time
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-01-18 11:14:45 -08:00
Akihiro Suda 6e7617e889 support local cache exporter and importer
Export:

  $ buildctl build ... --export-cache type=local,store=/path/to/output-dir

Import:

  $ buildctl build ... --import-cache type=local,store=/path/to/input-dir

Impact on CLI:
* Old (deprecated but still effective): `--export-cache localhost:5000/myrepo:buildcache --export-cache-opt mode=max`
* New: `--export-cache type=registry,ref=localhost:5000/myrepo:buildcache,mode=max`

Impact on API:
* New fields are added to control.proto and gateway.proto. The daemon
internally translates old API calls to the new ones.
* While new API can be used for `registry` caches, the client continues
to use the legacy API for `registry` caches to ensure compatibility with
old daemons.
* To import `local` caches with a frontend, the frontend needs to support
a new frontend opt `cache-imports`.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-01-19 02:38:26 +09:00
Tõnis Tiigi 34ff9c2366
Merge pull request #771 from LinuxMercedes/error-on-empty-dockerfile
Have parser error on dockerfiles without instructions
2019-01-04 16:33:08 -08:00
linuxmercedes 2ec7d53b00 Parser errors on dockerfiles without instructions
See https://github.com/moby/moby/pull/38487 for more discussion.

Signed-off-by: Natasha Jarus <linuxmercedes@gmail.com>
2019-01-04 13:44:52 -06:00
Tonis Tiigi b521aae3ea dockerfile: allow setting file mode/uid for secrets
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-01-03 11:13:40 -08:00
Tonis Tiigi 2529762ae7 dockerfile: allow setting file mode/uid for ssh socket
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-01-03 11:13:34 -08:00
Tonis Tiigi 3926820e3a update dockerfile repo location
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-12-03 11:06:17 +01:00
Wei Fu 2a7287a189 dockerfile2llb/pipe: remove the useless assignment
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2018-11-28 19:44:53 +08:00
Tonis Tiigi 2e3058e62c dockerfile: allow symlinks on reading Dockerfile
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-11-20 18:15:41 -08:00
Tibor Vass bf8c057e68
Merge pull request #727 from tonistiigi/add-fix
dockerfile: avoid urls in context filter
2018-11-20 16:47:31 -08:00
Tonis Tiigi b3e4cdf0d8 dockerfile: document more mount options
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-11-19 10:10:10 -08:00
Tonis Tiigi c840e7967f dockerfile: avoid urls in context filter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-11-18 12:36:26 -08:00
Tonis Tiigi 9d3426d94f dockerfile: update image refs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-11-07 17:51:02 -08:00
Tonis Tiigi ec2ab81a3b dockerfile: update copy source repo
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-11-06 11:14:59 -08:00
Akihiro Suda b5003d53eb update docs
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-11-01 16:17:26 +09:00
Akihiro Suda 03d15c8ff2 bump up tonistiigi/copy to v0.1.9
Fix #704 (CoreOS xattr issue)

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-31 13:30:26 +09:00
Tõnis Tiigi 85935a3d1e
Merge pull request #661 from tiborvass/dockerfile-shell-only-envvar
dockerfile: RUN's CustomName no longer consume quotes and only replaces environment variables if set
2018-10-29 11:04:07 -07:00
Tonis Tiigi 134e126aab dockerfile: add daily feature releases
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-23 22:43:21 -07:00
Tõnis Tiigi 50b36877f5
Merge pull request #695 from tonistiigi/dockerfile-release
dockerfile: add release tag deployments
2018-10-22 17:41:12 -07:00
Tonis Tiigi 945e100a8c dockerfile: disable network for copy
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-22 13:23:10 -07:00
Tonis Tiigi 918ee1988a dockerfile: add release tag deployments
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-17 14:11:09 -07:00
Tonis Tiigi 4c3050a26c dockerfile: setup master release deployment
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-16 21:01:33 -07:00
Akihiro Suda 0d80bd17a3
Merge pull request #687 from tonistiigi/dockerfile-testing
dockerfile: add testing external dockerfile features
2018-10-16 15:58:55 +09:00
Tonis Tiigi 12513ba394 dockerfile: remove dfextall build tag
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-15 22:27:13 -07:00
Tonis Tiigi 74754d6a6a dockerfile: add example mount test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-15 22:27:13 -07:00
Tonis Tiigi 706fa2556a dockerfile: add testing external dockerfile features
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-15 22:27:07 -07:00
Akihiro Suda 048130d1d0 simplify rootless
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-16 14:05:58 +09:00
Tonis Tiigi 94a6c5c79d dockerfile: add test for empty copy directory
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-15 16:43:21 -07:00
Tonis Tiigi 1f51f5ea11 testutil: avoid excessive daemons in integration tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-11 16:47:35 -07:00
Tonis Tiigi af8b839283 dockerfile: fix empty dest directory panic
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-11 11:24:02 -07:00
Tonis Tiigi 2f8c7667d8 dockerfile: wrap reference parse error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-08 23:15:10 -07:00
Akihiro Suda ab6a2dc83e
Merge pull request #669 from tonistiigi/update-copy
dockerfile: update default copy image
2018-10-06 11:11:49 +09:00
Akihiro Suda 98e26e9685
Merge pull request #668 from tonistiigi/ssh-required-opt
dockerfile: fix ssh required option
2018-10-06 06:50:27 +09:00
Tonis Tiigi 48ec136f1b dockerfile: update default copy image
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-05 14:35:33 -07:00
Tonis Tiigi 079b429c12 dockerfile: fix ssh required option
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-05 14:10:09 -07:00
Ian Campbell 2ea0269ec4 gateway: check for `ReadDir` and `StatFile` caps on client side
This will produce a "requested feature is not supported" style error message
with some more specific details about the build server configuration rather
than a more generic gRPC "unsupported" error message.

Ensure that the tests pass the caps to `Marshal` so they see the new
functionality.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-10-05 15:20:50 +01:00
Tõnis Tiigi 520201006c
Merge pull request #665 from tiborvass/fix-dockerfile-load-output
dockerfile: change output when loading Dockerfile
2018-10-04 19:22:00 -07:00
Tibor Vass ef99a677c5 dockerfile: change output when loading Dockerfile
Previously, the output said "load Dockerfile from " + filename when the
file is not named Dockerfile. With this patch, it is changed to always
show "load build definition from " + filename.

This is because `# syntax = ...` can be put in non-Dockerfiles.

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-10-04 20:52:15 +00:00
Ian Campbell b4454a169f implement `gateway.Reference.StatFile` method
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-10-03 12:16:45 +01:00
Ian Campbell df76f6d380 Move cache/fsutil into a separate cache/util subpackage
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-10-03 12:16:44 +01:00
Ian Campbell b30ee91a53 implement `gateway.Reference.ReadDir` method
The alternative is a helper container running e.g. find, capturing the output
to a scratch to be read back (with ref.ReadFile) and parsed.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-10-03 12:16:44 +01:00
Tõnis Tiigi c9dc8352a4
Merge pull request #656 from AkihiroSuda/cap-ssh
define CapMountSSH
2018-10-02 23:21:17 -07:00
Akihiro Suda 758a267235 s/CapMount/CapExecMount/g
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-03 15:03:36 +09:00
Akihiro Suda fa92effc4b dockerfile: add support for RUN --mount=type=ssh
* Needs to be compiled with `dfrunmount dfssh`
* Implemented options:
 * `type`(required): needs to be `ssh`
 * `target`(optional): the socket path in the container
 * `id`(optional): id

Test script:

	#!/bin/bash
	set -exu -o pipefail
	REF=localhost:5000/dfssh:latest
	ssh-add -l
	sudo buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=frontend/dockerfile/cmd/dockerfile-frontend \
	  --frontend-opt "build-arg:BUILDTAGS=dfrunmount dfssh" \
	  --exporter=image --exporter-opt name=$REF --exporter-opt push=true
	mkdir -p /tmp/foo
	cd /tmp/foo
	cat << EOF > Dockerfile
	# syntax=$REF
	FROM alpine
	RUN apk add --no-cache openssh-client
	RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan gitlab.com >> ~/.ssh/known_hosts
	RUN --mount=type=ssh ssh git@gitlab.com
	# "Welcome to GitLab, @GITLAB_USERNAME_ASSOCIATED_WITH_SSHKEY" should be printed here
	EOF
	sudo buildctl build --ssh default=$SSH_AUTH_SOCK --progress=plain --frontend=dockerfile.v0 --local context=. --local dockerfile=.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-03 14:53:43 +09:00
Tibor Vass b9859b2d49 dockerfile: RUN's CustomName no longer consume quotes and only replaces environment variables if set
```
FROM alpine
RUN echo "Hey $name"
```

When building the above Dockerfile, the RUN command used to show: `echo Hey `
With this patch it now shows `echo "Hey $name"`.

However, because it is very useful for debugging to see the values variables can take,
they are expanded if set.

```
FROM alpine
ENV name=Jude
RUN echo "Hey $name"
```

Building the above Dockerfile makes the RUN command show: `echo "Hey Jude"`.

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-10-03 02:41:51 +00:00
Tonis Tiigi d40d557692 dockerfile: fix building dfextall
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-02 10:33:18 -07:00
Tibor Vass 38198bd5c3 dockerfile: substitute build args in addition to env vars in RUN's CustomName
Example Dockerfile:

 FROM busybox
 ARG foo=abc
 ENV bar=def
 RUN echo $foo $bar $baz

Output without fix:

 #5 [2/2] RUN echo  def
 #5       digest: sha256:647df948c9689163efaf92b24b38779bc9f8b350482ecc3de3533f86a544c196
 #5         name: "[2/2] RUN echo  def "
 #5      started: 2018-09-29 19:01:11.376665368 +0000 UTC
 #5 0.764 abc def
 #5    completed: 2018-09-29 19:01:12.320141054 +0000 UTC
 #5     duration: 943.475686ms

Output with fix:

 #5 [2/2] RUN echo abc def
 #5       digest: sha256:647df948c9689163efaf92b24b38779bc9f8b350482ecc3de3533f86a544c196
 #5         name: "[2/2] RUN echo abc def "
 #5      started: 2018-09-29 19:01:11.376665368 +0000 UTC
 #5 0.764 abc def
 #5    completed: 2018-09-29 19:01:12.320141054 +0000 UTC
 #5     duration: 943.475686ms

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-09-29 19:07:01 +00:00
Tonis Tiigi 25bfc57e9e dockerfile: add test for chown destination
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-25 23:10:17 -07:00
Tõnis Tiigi e04e16c836
Merge pull request #641 from tonistiigi/llb-caps-check
gateway: allow access to apicaps
2018-09-25 09:02:40 -07:00
Tonis Tiigi 895950cecf executor: fix submount symlink resolution
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-24 17:42:38 -07:00
Tonis Tiigi f6e104da5f gateway: allow access to apicaps
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-21 15:33:41 -07:00
Tonis Tiigi f6352a305f dockerfile: add symlink copy tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-21 13:16:57 -07:00
Tõnis Tiigi 39404586a5
Merge pull request #622 from tonistiigi/df-modes-switch
dockerfile: enable frontend selection in integration tests
2018-09-20 14:44:11 -07:00
Tibor Vass 9890dda814
Merge pull request #634 from tonistiigi/schema1-config-fix
imageutil: fix getting schema1 configs
2018-09-19 15:50:52 -07:00
Tonis Tiigi 7615f02078 dockerfile: avoid detecting schema1 configs as empty
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-19 14:09:37 -07:00
Tonis Tiigi 2ab0922500 dockerfile: enable frontend selection in integration tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-18 22:07:38 -07:00
Tonis Tiigi a75983ddb9 integration: pass mirrored images from test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-18 22:07:38 -07:00
Tonis Tiigi 0940cdc6fe update golint comments
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-18 22:06:47 -07:00
Tibor Vass 0cd2b56a52
Merge pull request #629 from tonistiigi/df-output-fix
dockerfile: improve output
2018-09-17 15:39:14 -07:00
Tonis Tiigi 3938aebdfc llb: make sure to use stable marshaler
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-17 15:19:04 -07:00
Tonis Tiigi 705b5f0a4a dockerfile: fix command count & platform for scratch stages
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-17 13:42:25 -07:00
Tonis Tiigi 9c6eb52a84 dockerfile: avoid duplicate config resolve lines
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-17 13:42:21 -07:00
Tonis Tiigi b103d7fdd6 dockerfile: allow relative paths in mount targets
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-17 11:15:22 -07:00
Akihiro Suda bf571a519e update Go to 1.11
For consistency with Moby (moby/moby#37358)

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-09-15 14:25:11 +09:00
Tonis Tiigi c1d0aa7c5a dockerfile: set default PATH for arg replacement
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-11 12:15:44 -07:00
Tõnis Tiigi 9a6e2a5a91
Merge pull request #607 from kunalkushwaha/fix-typo-1
function name fixed SetMarhalDefaults -> SetMarshalDefaults
2018-09-05 22:01:39 -07:00
Kunal Kushwaha 98c1f44151 function name fixed SetMarhalDefaults -> SetMarshalDefaults
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-09-06 13:28:39 +09:00
Akihiro Suda 756ca6cc6d
Merge pull request #602 from tonistiigi/gc
automatic GC
2018-09-05 10:47:21 +09:00
Tonis Tiigi 079c747c4e ontrol: support calling automatic GC
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-04 18:13:29 -07:00
Tibor Vass 6812dac65e
Merge pull request #596 from tonistiigi/required-secret
dockerfile: add required option for secrets
2018-09-04 08:05:41 -07:00
Tonis Tiigi 372528a565 dockerfile: update copy helper to v0.1.4
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-31 14:44:08 -07:00
Tibor Vass 94b009b27e
Merge pull request #594 from tonistiigi/override-copy
dockerfile: allow overriding copy image
2018-08-31 14:35:06 -07:00
Tõnis Tiigi e3bc522f7c
Merge pull request #582 from ijc/gateway-client-snapshot-leak
clientside-frontend: release non-result refs after completion
2018-08-31 09:22:28 -07:00
Tonis Tiigi 3eda0516da dockerfile: allow overriding copy image
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-30 13:21:52 -07:00
Ian Campbell 53921cf110 llbBridfgeForwarder: Drop refs as they are released in `Discard`.
This makes it safe(r) for `Discard` to be called more than once.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-30 15:23:49 +01:00
Ian Campbell b558d75d3a Consistently protect `llbBridgeForwarder.refs` with mutex
Most accesses to this map are protected. Add the missing locks (some of which
are in code paths where we wouldn't normally expect concurrent accesses, but
better to be safe).

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-30 15:03:29 +01:00
Ian Campbell 59391d1647 clientside-frontend: release non-result refs after completion
Tests such as TestNoSnapshotLeak were failing in client mode (e.g. using #522)
because we weren't releasing the intermediate refs.

Resolve this by refactoring the existing code which frees the intermediate refs
from `gatewayFrontend.Solve` into a method on `llbBridgeForwarder` and as well
as the original site also call from the solver when the top-level solve (in
clientside frontend mode) completes. The original call (which is via a defer)
could likely sensibly be moved either earlier or later if desired but leave it
here it is to minimise the scope of the change.

The previous code used the `retErr` named return but the code between that
point and the end of the function already ensured that `lbf.err` is the same as
`retErr`, thus the only change in the code which has moved is
`s/retErr/lbf.err/`. Note that the `res` named return was previously unused by
name.

Fixes #581.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-30 14:54:50 +01:00
Tibor Vass 17e6416706
Merge pull request #504 from kunalkushwaha/no-console
--no-progress replaced with --progress
2018-08-29 13:44:51 -07:00
Tonis Tiigi 6daf696f04 dockerfile: add required option for secrets
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-28 16:18:08 -07:00
Tonis Tiigi 38569b9b9a dockerfile: never use entrypoint for run commands
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-27 17:48:18 -07:00
Tonis Tiigi 4963ed7ea4 dockerfile: fix quoting of meta args
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-27 09:39:55 -07:00
Kunal Kushwaha 49c9f75275 --no-progress replaced with --progress
this make buildctl build output options same as moby.

Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-08-27 11:23:15 +09:00
Akihiro Suda decb27618b containerd: set timeout explicitly (60s)
The contained default value was shortened from 60s to 10s in containerd/containerd#2554

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-08-23 18:14:57 +09:00
Akihiro Suda af46188e9b
Merge pull request #533 from ijc/client-gateway
access gateway API from client
2018-08-17 05:12:23 +09:00
Ian Campbell 22f632f895 allow frontends to be run directly on the control client side
This allows builder code to be written which can be built as either a gateway
container or in a purely client side configuration, giving implementors more
flexibility.

Now when `Solve` sees a request with neither a definition nor a frontend
specified it will make the job available via new LLBBridge endpoints on the
control socket which the client can then use. These end points require the job
id to be present in the gRPC metadata and a client side object is added to
facilitate this.

The `llbBridgeForwarder` type is now exposed as a public `interface
LLBBridgeForwarder` which satisfies the underlying gRPC server interface
(`pb.LLBBridgeServer`) as well as a new `Done()` & `Result()` pair which can be
used to wait for the client to call `Return()` (using a model similar to
`context.Context`).

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-14 11:50:08 +01:00
Tonis Tiigi 95e0348f57 dockerfile: enable global net-mode settings
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-09 14:03:40 -07:00
Akihiro Suda 809b9a4602
Merge pull request #568 from tonistiigi/multi-platform
dockerfile: allow multi-target external builds
2018-08-10 00:09:35 +09:00
Ian Campbell 2a4ec0b192 gateway: call `Return` if call to `BuildFunc` returns an error.
Reorder the code slightly so that an error returned by the call to the
`BuildFunc` is picked up by the `defer`d error handling.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-09 11:19:42 +01:00
Ian Campbell 13f53d5753 gateway: tolerate `Solve` being called with no `creq.Definition`
Avoids panicing when accessing `creq.Definition.Metadata`.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-09 11:19:42 +01:00
Ian Campbell 3f5276b3c4 gateway: make `Return` a one shot operation and check for various edge cases.
Refactor the setting/getting of the result for a `llbBridgeForwarder` in order
to check that `Return` is only called once and only with the correct options.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-09 11:19:42 +01:00
Ian Campbell 5485309658 gateway: Return an `interface` from `grpcclient.New`.
Returning an interface rather than a private-struct from a public interface is
good practice.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-09 11:19:29 +01:00
Tonis Tiigi fa83e55b21 dockerfile: allow multi-target external builds
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-08 18:42:43 -07:00
Tonis Tiigi 5e6729e56f dockerfile: implement secret type mounts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-08 18:12:16 -07:00
Ian Campbell 682dce615a gateway: support direct creation/running of a client.
Provide a `New` method and a new `Run` method on the `grpcClient` to allow
manual creation of a client in addition to the current ability to initialise
from the environment. Accordingly the existing `Run` method becomes
`RunFromEnvironment`.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-08 10:17:32 +01:00
Tonis Tiigi f8b0573edb dockerfile: expose add-hosts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-03 13:56:14 -07:00
Tonis Tiigi 04a5cc81e8 dockerfile: add platform prefixes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-31 10:41:36 -07:00
Tonis Tiigi d0aafaefd0 dockerfile: add stage prefixes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-31 10:41:36 -07:00
Tonis Tiigi 4c44d2c6e4 dockerfile: custom names to vertexes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-31 10:41:30 -07:00
Tonis Tiigi 97fa2b527a cache: add all option to prune
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-26 22:55:05 -07:00
Tonis Tiigi 489246dd28 cache: support for internal/frontend record type
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-26 22:54:53 -07:00
Tonis Tiigi b966096ab5 dockerfile: allow setting image resolve mode
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 18:23:59 -07:00
Tonis Tiigi 9f5e5c7dc7 metaresolver: refactor to option struct and resolvemode
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 18:22:15 -07:00
Tonis Tiigi f719aadcba dockerfile: add tests for multi-platform exports
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 16:01:12 -07:00
Tonis Tiigi 6be1257f5d export: manifest list support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 16:01:12 -07:00
Tonis Tiigi 8c065401f2 exporter: reuse exptypes constants
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 16:01:12 -07:00
Tonis Tiigi 69bb8e8a15 dockerfile: allow automatic platform variables
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 16:01:05 -07:00
Tonis Tiigi 33f4382b69 dockerfile: support exporting multiple platforms
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 14:37:01 -07:00
Tonis Tiigi 0d6b052c97 dockerfile: add test for incremental copy regression
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-24 23:28:55 -07:00
Yuichiro Kaneko 08a5847eff Split resolving platform codes to `buildPlatformOpt`
This will make it easy to test these functions.

Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-24 08:46:32 +09:00
Yuichiro Kaneko 85987b10ee Rename `toEnvList` to `toEnvMap`
Because `toEnvList` was changed to return map.

Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-21 08:30:47 +09:00
Yuichiro Kaneko 80e2176450 Omit needless argument `override`
It always called with `override = true`.

Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-20 19:46:50 +09:00
Yuichiro Kaneko 936aa64061 Use `ProcessWord(s)WithMap` to avoid string contatination and string split
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-20 19:41:15 +09:00
Yuichiro Kaneko 5b70c51b7d Add test cases for `toEnvList`
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-20 19:31:43 +09:00
Yuichiro Kaneko 75118c8a82 Add Map version `ProcessWords`
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-20 19:01:31 +09:00
Tibor Vass 71e3c24df1
Merge pull request #517 from tonistiigi/llb-caps
llb: add caps support
2018-07-19 17:21:50 -07:00
Yuichiro Kaneko 83dc829222 Extract parsing logic of "key=val" as `parseKeyValue`
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-19 21:36:07 +09:00
Tõnis Tiigi db8b382274
Merge pull request #518 from yui-knk/add_test_for_addEnv
Add test cases for `addEnv` which is a bit complicated function
2018-07-18 22:41:39 -07:00
Akihiro Suda 1be4a5891d
Merge pull request #516 from yui-knk/update_comment_Parse
Add a description about metaArgs return value
2018-07-19 12:16:37 +09:00
Yuichiro Kaneko f79f28cf83 Add test cases for `addEnv` which is a bit complicated function
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-19 09:39:22 +09:00
Tonis Tiigi c8cb8127c3 gateway: validate LLB capabilities
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-18 17:22:30 -07:00
Tonis Tiigi d4c57aec5d llb: add caps support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-18 17:22:30 -07:00
Yuichiro Kaneko 93e3b15854 `s/docker file/Dockerfile/`
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-19 07:38:58 +09:00
Yuichiro Kaneko ee15340993 Add a description about metaArgs return value
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-18 22:32:15 +09:00
Yuichiro Kaneko fb499edc5b Omit needless argument `commit` from `dispatchEnv`
Always `dispatchEnv` is called `commit` with `true`.

Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-18 21:38:37 +09:00
Yuichiro Kaneko 5838f9e763 Simplify `State` setup
On this code `dispatchEnv` is called with `commit = false`,
so this call do these 2 things

* `AddEnv` to `d.state`
* `addEnv` to update `d.image.Config.Env`

But updating `d.image.Config.Env` has no effect on `d.image.Config.Env`
because this is in for-loop of `range d.image.Config.Env`.

Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-18 21:29:55 +09:00
Yuichiro Kaneko c9038a2f5d Use `ProcessWordWithMap` in simple cases
In the case when the second argument of `toEnvList` is `nil`,
we can use `ProcessWordWithMap` and avoid string contatination
and string split.

Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-18 08:54:18 +09:00
Yuichiro Kaneko 8715fbf6bf Add string map version ProcessWord to Lex
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-16 17:49:28 +09:00
Yuichiro Kaneko 6583105db9 Use strings map instead of slice of string in `shellWord`
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-16 17:46:28 +09:00
Tonis Tiigi 303b5da713 llbsolver: use result type with multi ref support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-12 17:52:23 -07:00
Tonis Tiigi c3a29cce8b gateway: implementation for return
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-12 17:51:44 -07:00
Tonis Tiigi 95d944cc31 gateway: use return in grpcclient
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-12 17:51:44 -07:00
Tonis Tiigi 01b7a29e86 gateway: return result from solve
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-12 17:51:44 -07:00
Tonis Tiigi 9684362cb0 frontend: move forwarder under gateway
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-12 17:50:19 -07:00
Tonis Tiigi d9f1a1d99f gateway: expose buildopts and result
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-12 17:50:19 -07:00
Tonis Tiigi 30601fb7e5 gateway: fix product env passing
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-12 16:04:58 -07:00
Tonis Tiigi bdf391a67d dockerfile: test for snapshot leak
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-11 16:21:32 -07:00
Tonis Tiigi ddf3f3e59c frontend: fix two data races
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-10 19:31:33 -07:00
Tõnis Tiigi f4694ee01b
Merge pull request #488 from yui-knk/stop_create_ArgCommand
Use simple `keyValuePair` to `dispatchOpt.metaArgs` and `dispatchState.buildArgs`
2018-07-10 08:02:21 -07:00
Yuichiro Kaneko c641b43e4a Use simple `KeyValuePairOptional` to `dispatchOpt.metaArgs` and `dispatchState.buildArgs`
`KeyValuePairOptional` is enough in this case. We can avoid create `ArgCommand` out of Parser
 by replacing them with `KeyValuePairOptional`.

Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-10 19:24:43 +09:00
Tonis Tiigi d5c9d39128 apicaps: set exportedproduct from binary
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-09 16:32:39 -07:00
Tonis Tiigi c8fa75ffc1 dockerfile: read default build platform from workers
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-09 16:32:39 -07:00
Tonis Tiigi 946a12eb42 gateway: expose worker information
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-09 16:32:39 -07:00
Tonis Tiigi 90d164ced5 gateway: inital caps detection support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-09 16:32:38 -07:00
Tonis Tiigi ae1127be14 frontend: define default caps
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-09 16:32:38 -07:00
Tonis Tiigi db5b0f1b9a api: add caps support for versioning
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-09 16:32:34 -07:00
Tõnis Tiigi cda0a64eca
Merge pull request #486 from AkihiroSuda/update-rootless
rootless: refactor libcontainer_specconv & add integration tests
2018-07-05 11:26:26 -07:00
Tõnis Tiigi c111113a2f
Merge pull request #278 from AkihiroSuda/cache-transferable
decouple remotecache from Docker registry
2018-07-05 11:18:50 -07:00
Yuichiro Kaneko b5c66082a1 Fix orders of `require.Equal` argument to "expected, actual"
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-04 21:59:37 +09:00
Akihiro Suda 580dbe08b9 rootless: add integration tests
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-07-04 19:27:54 +09:00
Akihiro Suda 80d2f820f9 decopule cache/remotecache from registry
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-07-04 17:38:09 +09:00
Yuichiro Kaneko 30c27cd6bf Return an error when copying from url
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-03 20:28:08 +09:00
Yuichiro Kaneko ec798d6b09 Update documents of `AddCommand`
`ADD` does not support git.

Ref: https://github.com/moby/moby/issues/14704#issuecomment-298134333

Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-03 09:25:30 +09:00
Yuichiro Kaneko b8bf5d0da9 Refactoring `toCommand` and `detectRunMount` to only depend to `dispatchStates`
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-03 07:38:34 +09:00
Yuichiro Kaneko 0d5c12363a Introduce `dispatchStates` type
`dispatchStates` manages a collection of `dispatchState`.
By `dispatchStates`, we can aggregate how to find `dispatchState`
in one place and `dispatchStatesByName` local variable in `Dockerfile2LLB`
can be removed.

Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-03 07:36:07 +09:00
Yuichiro Kaneko fdfa7b8431 Add description about Unix
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-07-01 00:08:35 +09:00
Yuichiro Kaneko a448c8f3fb Fix a typo
Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-06-30 11:20:33 +09:00
Anda Xu 7f64188f17 add missing supplementary group IDs
Signed-off-by: Anda Xu <anda.xu@docker.com>
2018-06-29 18:50:03 -07:00
Tõnis Tiigi 3b9737dd5a
Merge pull request #470 from yui-knk/ensure_arg_is_not_blank
Ensure BaseName of Stage is not blank
2018-06-28 10:32:11 -07:00
Yuichiro Kaneko 813575f62a Ensure BaseName of Stage is not blank
Related: https://github.com/moby/moby/issues/37325

Signed-off-by: Yuichiro Kaneko <spiketeika@gmail.com>
2018-06-28 20:21:41 +09:00
Akihiro Suda cce2080ddb
Merge pull request #468 from tonistiigi/update-containerd
vendor: update containerd to 08f7ee982
2018-06-27 14:37:47 +09:00
Tonis Tiigi 97abda7528 dockerfile: use manifest list copy image
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-26 21:22:32 -07:00
Tonis Tiigi 5dea4ade69 dockerfile: allow setting target platform with opts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-26 21:14:52 -07:00
Tonis Tiigi 65e6d4d21a dockerfile: allow autodetect platform from image config
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-26 10:42:30 -07:00
Tonis Tiigi e618878451 dockerfile: platform support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-25 11:06:08 -07:00
Tonis Tiigi f9383c48a3 source: image source platform support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-25 10:39:17 -07:00
Tonis Tiigi dc9de85069 llb: force platform in llb and allow constraints
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-22 16:49:03 -07:00
Akihiro Suda b36d2dc272
Merge pull request #459 from tonistiigi/optional-digest
gateway: allow digest to be optional
2018-06-21 08:42:42 +09:00
Tonis Tiigi bdb77f073a gateway: protect gateway forwarder map access
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-20 11:17:05 -07:00
Tonis Tiigi e5449521f5 gateway: allow digest to be optional
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-20 11:09:32 -07:00
Akihiro Suda cfe4e19158
Merge pull request #455 from tonistiigi/cache-sharing
llbsolver: allow sharing modes for cache mounts
2018-06-21 00:17:05 +09:00
Tonis Tiigi cdcab49bfc dockerfile: expose cache sharing options
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-19 15:57:25 -07:00
Tonis Tiigi ccbf185006 llbsolver: allow sharing modes for cache mounts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-19 15:57:20 -07:00
Tonis Tiigi 941407f6c0 dockerfile: escape git regexp
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-18 11:26:07 -07:00
Akihiro Suda 00140b7992
Merge pull request #440 from ijc/configurable-client-timeout
client: take a context.Context to New().
2018-06-15 16:08:33 -07:00
Vincent Demeester 79241f25d5
Update tests to use gotest.tools 👼
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-11 15:52:17 +02:00
Ian Campbell 97ffd6cce8 client: take a context.Context to New().
This allows two things:

- The caller to set a shorter timeout than previously hardcoded 30s. In
  `buildctl` reduce the timeout to 5s. Since the existing timeout has gone
  callers will need to arrange to pass one themselves.
- The caller can arrange for the context to be cancelled for other reasons, use
  this in `buildctl` to plumb through the Ctrl-C handling, meaning that
  `buildctl` now exits almost immediately on Ctrl-C instead of after several
  seconds.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-06-11 14:28:08 +01:00
Tõnis Tiigi 704a40048b
Merge pull request #448 from tiborvass/update-copy-image
dockerfile: update copy image to v0.1.3
2018-06-08 20:06:17 -07:00
Tibor Vass eb44845441 dockerfile: update copy image to v0.1.3
Signed-off-by: Tibor Vass <tibor@docker.com>
2018-06-09 02:49:34 +00:00
Tonis Tiigi 6f569455d0 dockerfile: enable tmpfs mounts on run —mount
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-08 13:42:38 -07:00
Akihiro Suda 60344aa592
Merge pull request #442 from tonistiigi/runmount
dockerfile: add run --mount support
2018-06-09 02:35:34 +09:00
Tõnis Tiigi 7943598281
Merge pull request #439 from AkihiroSuda/vendor-containerd-20180606
update containerd library
2018-06-08 10:33:10 -07:00
Akihiro Suda 47a00b84fe update containerd library
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-06-08 21:44:17 +09:00
Tonis Tiigi af03a526e7 dockerfile: run -—mount initial support
Supports binds from images and context and cache mounts.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-07 16:23:40 -07:00
Tonis Tiigi aeea615e85 dockerfile: add run mount parsing
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-07 14:55:13 -07:00
Tonis Tiigi 103cd072d6 dockerfile: detect source symlinks with targets
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-05 23:54:13 -07:00
Tonis Tiigi 76d15da0d5 dockerfile: update imports
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-01 20:10:18 -07:00
Sebastiaan van Stijn 7d6f79781d Handle multi-digit positional parameters
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

rewritten from github.com/moby/moby 2628896b5e813491f8767ec7fa9d0f057ed4a86e
2018-06-01 20:10:13 -07:00
Sebastiaan van Stijn 13cd59db64 Add detection of "special parameters" for substitution
Detect Special parameters as defined in
http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_05_02

Treat these as parameters that are not set, instead of
producing an error that a modifier is missing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

rewritten from github.com/moby/moby 9654e9b6f80e1b931763c04400a19596e256c99a
2018-06-01 20:10:13 -07:00
Sebastiaan van Stijn cd7c16b606 Add more test-cases for positional parameters
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

rewritten from github.com/moby/moby 8687a3f4b8b7ffbc3f32dfe959cb771d662211e6
2018-06-01 20:10:13 -07:00
Sebastiaan van Stijn 5e7c39f879 Add line-numbers to asserts
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

rewritten from github.com/moby/moby b80e0309d220268a2b9e6aec5bb05d7af330e591
2018-06-01 20:10:13 -07:00
Sebastiaan van Stijn e15e0ddbf5 Fix detection for missing parameter in substitution
`${}`, `${:}` and so on are invalid because there's
no parameter within the brackets; fix detection for
this situation and add/update tests.

There were some existing test-cases that were testing
for the wrong behavior, which are now updated.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

rewritten from github.com/moby/moby 334bf3ea76004d0abe02dd1698989f9eaf87a86a
2018-06-01 20:10:13 -07:00
Sebastiaan van Stijn 33114da1da builder: fix processing of invalid substitusion syntax
The builder did not detect syntax errors in substitusions in the
Dockerfile, causing those values to be processed incorrectly instead of
producing an error.

Example 1: missing `}`

    docker build --no-cache -<<'EOF'
    FROM busybox
    ARG var=${aaa:-bbb
    RUN echo $var
    EOF

Before:

    Step 3/3 : RUN echo $var
     ---> Running in f06571e77146
    bbb

After:

    Step 2/3 : ARG var=${aaa:-bbb
    failed to process "${aaa:-bbb": syntax error: missing '}'

Example 2: missing closing `}`, no default value

    docker build --no-cache -<<'EOF'
    FROM busybox
    ARG var=${aaa
    RUN echo $var
    EOF

Before:

    Step 2/3 : ARG var=${aaa
    failed to process "${aaa": missing ':' in substitution

After:

    Step 2/3 : ARG var=${aaa
    failed to process "${aaa": syntax error: missing '}'

Example 3: double opening bracket (`{`)

    docker build --no-cache -<<'EOF'
    FROM busybox
    ARG var=${{aaa:-bbb}
    RUN echo $var
    EOF

Before:

    Step 2/3 : ARG var=${{aaa:-bbb}
    failed to process "${{aaa:-bbb}": missing ':' in substitution

After:

    Step 2/3 : ARG var=${{aaa:-bbb}
    failed to process "${{aaa:-bbb}": syntax error: bad substitution

Example 4: double opening bracket (`{`), no default value

    docker build --no-cache -<<'EOF'
    FROM busybox
    ARG var=${{aaa}
    RUN echo $var
    EOF

Before:

    Step 2/3 : ARG var=${{aaa}
    failed to process "${{aaa}": missing ':' in substitution

After:

    Step 2/3 : ARG var=${{aaa}
    failed to process "${{aaa}": syntax error: bad substitution

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

rewritten from github.com/moby/moby 955a6ad95f7891a45692d975793abf1eeb07cdd5
2018-06-01 20:10:13 -07:00
Sebastiaan van Stijn 9dc3c39c2c Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

rewritten from github.com/moby/moby f23c00d8701e4bd0f2372a586dacbf66a26f9a51
2018-06-01 20:10:13 -07:00
Sebastiaan van Stijn 5d7a878e53 Replace deprecated testutil.ErrorContains()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

rewritten from github.com/moby/moby 55bebbaecf5e40db9d83b28080ce08dc8642f407
2018-06-01 20:10:13 -07:00
Eli Uriegas 30a794fa73 Remove references to old release process
This hasn't been the way to release Docker for the past year so let's
just remove them altogether

Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>

rewritten from github.com/moby/moby 80bc8b4766cc808186af91930152d78afb22aef8
2018-06-01 20:10:13 -07:00
Dennis Chen 49550b05a5 Remove unused 'label' related functions
Since we use `NewLabelCommand()` instead of `addNodesForLabelOption()`
to create the 'LABEL' commands from '--label' options, so all the related
functions should be removed.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>

rewritten from github.com/moby/moby c7b543164daed58fbea36471592438b4e53ab748
2018-06-01 20:10:13 -07:00
Dennis Chen c35701d68d Add 'LABEL' command from '--label' to the last stage
This PR is tring to fix issue #36996.

Currently for multi-stage build, if `--target` specified, the `--label` option
will be ignored. The root cause is the last stage build will remove the `LABEL`
command(s) node created from the `--label` option. In order to address this issue,
we can create `LABEL` command(s) and add it/tem to the last stage.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>

rewritten from github.com/moby/moby 9c238ebd55e4105ad7f7edc04231ea61bb278ae8
2018-06-01 20:10:13 -07:00
Dennis Chen 29887cc5b2 Fix the target name issue for multi-stage build
This PR is trying to fix issue #36956.

The stage name is case-insensitive by design, so we should use
`strings.EqualFold()` as the comparison method to eliminate the
case sensitive noise.

Also we need to return a pre-defined error code order to avoid below
message like:
"FIXME: Got an API for which error does not match any expected type!!!:
failed to reach build target dev in Dockerfile"

Signed-off-by: Dennis Chen <dennis.chen@arm.com>

rewritten from github.com/moby/moby 7c0570473cfa181aeb3278072cc9af4f9298cb98
2018-06-01 20:10:13 -07:00
John Howard 668e4abfbb Builder: Review feedback
Signed-off-by: John Howard <jhoward@microsoft.com>

rewritten from github.com/moby/moby 14429056d3745ca052fba448d879788d16bbb01b
2018-06-01 20:10:13 -07:00
John Howard 354aaeb223 Builder: Plumbing through platform in `FROM` statement
Signed-off-by: John Howard <jhoward@microsoft.com>

rewritten from github.com/moby/moby 69fa84bc3d57dafd19800642c5ba196bc6d45f90
2018-06-01 20:10:13 -07:00
John Howard 0381374837 Builder - add --platform to FROM statement
Signed-off-by: John Howard <jhoward@microsoft.com>

rewritten from github.com/moby/moby 7f0c2d23e11485c7f026dd8c111c60c2e1e03375
2018-06-01 20:10:13 -07:00
John Howard 9f97edb9e4 Builder - Parser. Remove platform parser directive
Signed-off-by: John Howard <jhoward@microsoft.com>

rewritten from github.com/moby/moby 9cae03900fc27ff39e913978ca8f084691954881
2018-06-01 20:10:13 -07:00
John Howard 99f656e7fd Builder - parser - remove OS
Signed-off-by: John Howard <jhoward@microsoft.com>

rewritten from github.com/moby/moby 98dd1fdca1f5b82cbc7066c4a48f9ddd8f135095
2018-06-01 20:10:13 -07:00
Daniel Nephin 414a98c2e0 Post migration assertion fixes
Signed-off-by: Daniel Nephin <dnephin@docker.com>

rewritten from github.com/moby/moby c9e52bd0da0461e605a3678b85702f83081504a7
2018-06-01 20:10:13 -07:00
Daniel Nephin 3b28b903b9 Automated migration using
gty-migrate-from-testify --ignore-build-tags

Signed-off-by: Daniel Nephin <dnephin@docker.com>

rewritten from github.com/moby/moby 6be0f709830113966f295401327b027ec2f0bbca
2018-06-01 20:10:13 -07:00
bin liu 0873a108bb Fix typos in builder and client
Signed-off-by: bin liu <liubin0329@gmail.com>

rewritten from github.com/moby/moby ce0993270436c16a17e0afb08b9130556346b983
2018-06-01 20:10:13 -07:00
Daniel Nephin a3b7488b81 Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>

rewritten from github.com/moby/moby 4f0d95fa6ee7f865597c03b9e63702cdcb0f7067
2018-06-01 20:10:13 -07:00
Matt Rickard 1f5fd4bb26 Move builder shell parser into subpackage
Moves builder/shell_parser and into its own subpackage at builder/shell since it
has no dependencies other than the standard library. This will make it
much easier to vendor for downstream libraries, without pulling all the
dependencies of builder/.

Fixes #36154

Signed-off-by: Matt Rickard <mrick@google.com>

rewritten from github.com/moby/moby a634526d14639c6b98c509a069ef29e2b69c0ef0
2018-06-01 20:10:13 -07:00
Anthony Sottile 0d95b8138b Change references from test-integration-cli to test-integration
Signed-off-by: Anthony Sottile <asottile@umich.edu>

rewritten from github.com/moby/moby b6eed3dca2e57705cb75895cad0b5337c79686d9
2018-06-01 20:10:13 -07:00
Daniel Nephin 2dc37b2ae6 Fix dockerfile parser failing silently on long tokens
Signed-off-by: Daniel Nephin <dnephin@docker.com>

rewritten from github.com/moby/moby 59ad3a36e2684bd36a4b02179949bd17f1406918
2018-06-01 20:10:13 -07:00
Ri Xu 182b2d5bd6 Typo fixed and simple code.
Signed-off-by: Ri Xu <xuri@360.net>

rewritten from github.com/moby/moby 87e8a936e82ecfbef59b829d8bbfca3fb4aa3163
2018-06-01 20:10:13 -07:00
John Howard ff49463d78 Fix TestErrorCases to run all tests
Signed-off-by: John Howard <jhoward@microsoft.com>
Signed-off-by: Vincent Demeester <vincent@sbr.pm>

rewritten from github.com/moby/moby d2c41f06c36dc24b2170fef365563d64cc5bd8ec
2018-06-01 20:10:13 -07:00
John Howard c7f561d8fa LCOW: API: Add platform to /images/create and /build
Signed-off-by: John Howard <jhoward@microsoft.com>

This PR has the API changes described in https://github.com/moby/moby/issues/34617.
Specifically, it adds an HTTP header "X-Requested-Platform" which is a JSON-encoded
OCI Image-spec `Platform` structure.

In addition, it renames (almost all) uses of a string variable platform (and associated)
methods/functions to os. This makes it much clearer to disambiguate with the swarm
"platform" which is really os/arch. This is a stepping stone to getting the daemon towards
fully multi-platform/arch-aware, and makes it clear when "operating system" is being
referred to rather than "platform" which is misleadingly used - sometimes in the swarm
meaning, but more often as just the operating system.

rewritten from github.com/moby/moby 0380fbff37922cadf294851b1546f4c212c7f364
2018-06-01 20:10:13 -07:00
Sebastiaan van Stijn 4afd7bf50e Warn on empty continuation lines only, not for comments
Commit 8d1ae76dcbbb73d8e20c6a14a7d3fe2410b95f55 added
deprecation warnings for empty continuation lines,
but also treated comment-only lines as empty.

This patch distinguishes empty continuation lines
from comment-only lines, and only outputs warnings
for the former.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

rewritten from github.com/moby/moby 2fd736ac10c1c46d1001373d887cb99b3d8ee824
2018-06-01 20:10:13 -07:00
Sebastiaan van Stijn 3b355e7083 Improve error message for COPY missing destination
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

rewritten from github.com/moby/moby 5d05a8291314b8f727b04b504b8d7fc7ed7f42da
2018-06-01 20:10:13 -07:00
Simon Ferquel 8cde317b4c Introduce a typed command system and 2 phase parse/dispatch build
This is a work base to introduce more features like build time
dockerfile optimisations, dependency analysis and parallel build, as
well as a first step to go from a dispatch-inline process to a
frontend+backend process.

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>

rewritten from github.com/moby/moby 669c0677980b04bcbf871bb7c2d9f07caccfd42b
2018-06-01 20:10:13 -07:00
Daniel Nephin 52c2623316 Add gosimple linter
Update gometalinter

Signed-off-by: Daniel Nephin <dnephin@docker.com>

rewritten from github.com/moby/moby f7f101d57ef8cbf2d8723a18b7d723c5c5dd04b6
2018-06-01 20:10:13 -07:00
Daniel Nephin 2a8909c525 Fix golint errors.
Signed-off-by: Daniel Nephin <dnephin@docker.com>

rewritten from github.com/moby/moby 9b47b7b1519c5f2138e2933fb1fc459eb00895c0
2018-06-01 20:10:13 -07:00
John Howard 0b8ce214e4 LCOW: Fix nits from 33241
Signed-off-by: John Howard <jhoward@microsoft.com>

rewritten from github.com/moby/moby 4ec9766a27ffb964cec6b8b2745725965b2644aa
2018-06-01 20:10:13 -07:00
Daniel Nephin 9def101b4a Add a test for warning on empty continuation lines.
Signed-off-by: Daniel Nephin <dnephin@docker.com>

rewritten from github.com/moby/moby b47b375cb8bb0dca7ee0ebfa093bfc163ad867fb
2018-06-01 20:10:13 -07:00
Yong Tang 9e0819d2a5 Warn/deprecate continuing on empty lines in `Dockerfile`
This fix is related to 29005 and 24693. Currently in `Dockerfile`
empty lines will continue as long as there is line escape before.
This may cause some issues. The issue in 24693 is an example.
A non-empty line after an empty line might be considered to be a
separate instruction by many users. However, it is actually part
of the last instruction under the current `Dockerfile` parsing
rule.

This fix is an effort to reduce the confusion around the parsing
of `Dockerfile`. Even though this fix does not change the behavior
of the `Dockerfile` parsing, it tries to deprecate the empty line
continuation and present a warning for the user. In this case,
at least it prompt users to check for the Dockerfile and avoid
the confusion if possible.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

rewritten from github.com/moby/moby 7815c8f8754d5473eda7cd80277a4ea3c59e3c29
2018-06-01 20:10:13 -07:00
John Howard 8a646a6115 LCOW: Builder plumbing
Signed-off-by: John Howard <jhoward@microsoft.com>

rewritten from github.com/moby/moby 08252bc963a20b2fb3052fc8db7143771c8c3248
2018-06-01 20:10:13 -07:00
Akihiro Suda a5bcc22c39 builder: add a test for `ENV name` (without `=value`)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>

rewritten from github.com/moby/moby 09eca591f723f9acd0909b04144c229b6cd59912
2018-06-01 20:10:13 -07:00
John Howard 1b72890ca2 Builder: Fix parser directive refactoring
Signed-off-by: John Howard <jhoward@microsoft.com>

rewritten from github.com/moby/moby c97170d618b7a1f35f47672143bd287666802287
2018-06-01 20:10:13 -07:00
Jean-Pierre Huynh 2b021f6760 MAINTAINER deprecation follow up. Relates to #25466
Signed-off-by: Jean-Pierre Huynh <jean-pierre.huynh@ounet.fr>

rewritten from github.com/moby/moby 2c435ab8beba93509b957e4ac5e5801b86b5e8f1
2018-06-01 20:10:13 -07:00
Aaron Lehmann b371d95625 Remove pkg/testutil/assert in favor of testify
I noticed that we're using a homegrown package for assertions. The
functions are extremely similar to testify, but with enough slight
differences to be confusing (for example, Equal takes its arguments in a
different order). We already vendor testify, and it's used in a few
places by tests.

I also found some problems with pkg/testutil/assert. For example, the
NotNil function seems to be broken. It checks the argument against
"nil", which only works for an interface. If you pass in a nil map or
slice, the equality check will fail.

In the interest of avoiding NIH, I'm proposing replacing
pkg/testutil/assert with testify. The test code looks almost the same,
but we avoid the confusion of having two similar but slightly different
assertion packages, and having to maintain our own package instead of
using a commonly-used one.

In the process, I found a few places where the tests should halt if an
assertion fails, so I've made those cases (that I noticed) use "require"
instead of "assert", and I've vendored the "require" package from
testify alongside the already-present "assert" package.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>

rewritten from github.com/moby/moby 6052f2b3969feadb01662d8e2f30337d9c7f61af
2018-06-01 20:10:13 -07:00
Daniel Nephin 8ff2f7277c Refactor dockerfile parser
Signed-off-by: Daniel Nephin <dnephin@docker.com>

rewritten from github.com/moby/moby 4cbc953a5d80c850df7107b28e743e933bbeb1d3
2018-06-01 20:10:13 -07:00
Daniel Nephin f36dfa3049 Cleanup processing of directives.
Signed-off-by: Daniel Nephin <dnephin@docker.com>

rewritten from github.com/moby/moby 9c53fa2d0c1796d32aaba2193e7d802cd9b66763
2018-06-01 20:10:13 -07:00
Daniel Nephin 36fc03783a Keep parser.Directive internal to parser
Signed-off-by: Daniel Nephin <dnephin@docker.com>

rewritten from github.com/moby/moby 64c4c1c3d5e0fe364d83db5a8dc99a24fd121754
2018-06-01 20:10:13 -07:00
Daniel Nephin 715c3b0ac6 Hide builder.parser.Directive internals
Signed-off-by: Daniel Nephin <dnephin@docker.com>

rewritten from github.com/moby/moby bb429da9a977a9d89121052fe1143528294b8a0c
2018-06-01 20:10:13 -07:00
Tonis Tiigi 6ac1bf9226 Add named context support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>

rewritten from github.com/moby/moby 87512bbc8490aca261133a7f3a3ea6518d703c34
2018-06-01 20:10:13 -07:00
Daniel Nephin 7510e5ed09 Fix --label being env var expanded.
Signed-off-by: Daniel Nephin <dnephin@docker.com>

rewritten from github.com/moby/moby 434d77bc0edf21b970e0414ee54112768a490aaa
2018-06-01 20:10:13 -07:00
Daniel Nephin a284c4bbe2 Fix --label on docker build when a single quote is used in the value
Signed-off-by: Daniel Nephin <dnephin@docker.com>

rewritten from github.com/moby/moby 726fb269cf004814bd7065ca2f29f6955746a9d9
2018-06-01 20:10:13 -07:00
Daniel Nephin 2387678317 Refactor some builder code
Signed-off-by: Daniel Nephin <dnephin@docker.com>

rewritten from github.com/moby/moby d7807c73169469c50e551704cc1957f798868f71
2018-06-01 20:10:13 -07:00
fate-grand-order e47bac49ce fix misspell in utils.go
Signed-off-by: Helen Xie <chenjg@harmonycloud.cn>

rewritten from github.com/moby/moby 2b79dbc89bf15c276145a0fa3d3728308349f86e
2018-06-01 20:10:13 -07:00
fate-grand-order f3e2ff6cdf use t.Fatal() to output the err message where the values used for formatting
text does not appear to contain a placeholder

Signed-off-by: Helen Xie <chenjg@harmonycloud.cn>

rewritten from github.com/moby/moby 2a8d6368d4a930203b93f75914173ab65bf3b0bc
2018-06-01 20:10:13 -07:00
Aaron.L.Xu ecb31e5da3 why there are so many mistakes in our repo (up to /cmd)
Signed-off-by: Aaron.L.Xu <likexu@harmonycloud.cn>

rewritten from github.com/moby/moby f8a40477960976f12dbbb72e5b133b5b0f05f9b3
2018-06-01 20:10:13 -07:00
Anthony Sottile b3d12d1e98 Remove unnecessary if in parser.Dump
Signed-off-by: Anthony Sottile <asottile@umich.edu>

rewritten from github.com/moby/moby 2283cd020373ebd00e5cf37a81df451349c3b506
2018-06-01 20:10:13 -07:00
Justin Cormack cd0e0661bb Remove sqlite
This drops support for migrations from pre-1.10 Docker versions, which
should be done via an external tool or an intermediate upgrade.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>

rewritten from github.com/moby/moby f8119bb7a76b5c42defb6e0a2dc67bd77ad29a5e
2018-06-01 20:10:13 -07:00
Sebastiaan van Stijn 02c53c03f2 Revert "Require continuation char to be last char in a line"
This reverts commit 105bc63295a7126798d3722a0e205c5ead4e2b1c,
which (although correct), resulted in a backward incompatible
change.

We can re-implement this in future, after this changes goes
through a deprecation cycle

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

rewritten from github.com/moby/moby c67550414b762ab32ea3e9645ebcafb79cb9059d
2018-06-01 20:10:13 -07:00
Vincent Demeester abafb74cda Revert "Fix dockerfile parser with empty line after escape"
This reverts commit 3e1b539e8d0ed4abf695b0a8c42346fba6d5a6b0.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>

rewritten from github.com/moby/moby 6e0666e622a8b1fc19c19eb490d218eebca15edc
2018-06-01 20:10:13 -07:00
Doug Davis 3a44018859 Fix case where \\ at EOF made the builder ignore the command
Came from looking at issue #27545

Signed-off-by: Doug Davis <dug@us.ibm.com>

rewritten from github.com/moby/moby eaf0b5708fc91ab0c78eeee1a2c9c1251439d4a8
2018-06-01 20:10:13 -07:00
Yong Tang f7172b5204 Fix dockerfile parser with empty line after escape
This fix tries to fix the bug reported by #24693 where an empty
line after escape will not be stopped by the parser.

This fix addresses this issue by stop the parser from continue
with an empty line after escape.

An additional integration test has been added.

This fix fixes #24693.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

rewritten from github.com/moby/moby 3e1b539e8d0ed4abf695b0a8c42346fba6d5a6b0
2018-06-01 20:10:13 -07:00
Doug Davis f6e18945e4 Require continuation char to be last char in a line
While look at #27039 I noticed that we allow for whitespace after
the continuation char (\\) which is wrong. It needs to be the very
last char in the line.

Signed-off-by: Doug Davis <dug@us.ibm.com>

rewritten from github.com/moby/moby 105bc63295a7126798d3722a0e205c5ead4e2b1c
2018-06-01 20:10:13 -07:00
allencloud 9eff79169e correct some nits in comments
Signed-off-by: allencloud <allen.sun@daocloud.io>

rewritten from github.com/moby/moby 9279a93f6d43da4c904eeb0adb249fdfa34f7f92
2018-06-01 20:10:13 -07:00
Thomas Leonard 44b4699319 Fix exec form of HEALTHCHECK CMD
We attached the JSON flag to the wrong AST node, causing Docker to treat
the exec form ["binary", "arg"] as if the shell form "binary arg" had
been used. This failed if "ls" was not present.

Added a test to detect this.

Fixes #26174

Signed-off-by: Thomas Leonard <thomas.leonard@docker.com>

rewritten from github.com/moby/moby e95b6b51daed868094c7b66113381d5088e831b4
2018-06-01 20:10:13 -07:00
allencloud c87514f875 add defer file.Close to avoid potential fd leak
Signed-off-by: allencloud <allen.sun@daocloud.io>

rewritten from github.com/moby/moby 0ead624473b6bddc232b46bc7c76ab4f9c743ff5
2018-06-01 20:10:13 -07:00
John Howard 834f4ed737 Move directive out of globals
Signed-off-by: John Howard <jhoward@microsoft.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

rewritten from github.com/moby/moby 755be795b4e48b3eadcdf1427bf9731b0e97bed1
2018-06-01 20:10:13 -07:00
Brian Goff 16e48eb857 Fix issue with test ordering for TestParseWords
`TestParseWords` needs to use the `tokenEscape` for one of the test
cases, but `tokenEscape` was not being set unless tests ran in a
specific order.
This sets a default value for `tokenEscape`... `\`... so that tests that
rely on this global are not affected by test ordering.

This is the simplest fix for these cases. Ideally the token should not
be set as a global but rather passed down, which is a much larger
change.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>

rewritten from github.com/moby/moby df167d3ff04cdc90012c8ca39647662ad69e6715
2018-06-01 20:10:13 -07:00
Shev Yan 969340ac31 Dockerfile's comments for "Run the test suite" are out of date because there is no 'test' in hack/make directory.
Signed-off-by: Shev Yan <yandong_8212@163.com>

rewritten from github.com/moby/moby 11280665b45d0ac9920d17abaf6a7328f2bc77d9
2018-06-01 20:10:13 -07:00
Tiffany Jernigan 3cde094e22 Fixed spelling error in builder/dockerfile/parser/parser.go
Signed-off-by: Tiffany Jernigan <tiffany.f.j@gmail.com>

rewritten from github.com/moby/moby a07c57f656846ee0de67ac4f575fe5b76d482efb
2018-06-01 20:10:13 -07:00
Jonathan Stoppani df7c901a02 Support unicode characters in parseWords
Signed-off-by: Jonathan Stoppani <jonathan.stoppani@divio.com>

rewritten from github.com/moby/moby 6284f04a6b3a9b8127dc166ff84b102c8ad411ce
2018-06-01 20:10:12 -07:00
John Howard 64323d3df2 Builder default shell
Signed-off-by: John Howard <jhoward@microsoft.com>

rewritten from github.com/moby/moby b18ae8c9ccc2eb6cf8aa947f25eb6f1d20089776
2018-06-01 20:10:12 -07:00
Yong Tang ed9a858fdd Skip UTF-8 BOM bytes from Dockerfile if exists
This fix tries to address issues in #23221 where Dockerfile
may consists of UTF-8 BOM. This likely happens when Notepad
tries to save a file as UTF-8 in Windows.

This fix skips the UTF-8 BOM bytes from the beginning of the
Dockerfile if exists.

Additional tests has been added to cover the changes in this
fix.

This fix fixes #23221.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

rewritten from github.com/moby/moby 678c80f9256021ce74184fdd6b612d9dea377fba
2018-06-01 20:10:12 -07:00
Sebastiaan van Stijn f6e17db76c Healthcheck: set default retries to 3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

rewritten from github.com/moby/moby 50e470fab4ebdffff74bafeefa2f7fb8c21da13e
2018-06-01 20:10:12 -07:00
Thomas Leonard e58dde3126 Add support for user-defined healthchecks
This PR adds support for user-defined health-check probes for Docker
containers. It adds a `HEALTHCHECK` instruction to the Dockerfile syntax plus
some corresponding "docker run" options. It can be used with a restart policy
to automatically restart a container if the check fails.

The `HEALTHCHECK` instruction has two forms:

* `HEALTHCHECK [OPTIONS] CMD command` (check container health by running a command inside the container)
* `HEALTHCHECK NONE` (disable any healthcheck inherited from the base image)

The `HEALTHCHECK` instruction tells Docker how to test a container to check that
it is still working. This can detect cases such as a web server that is stuck in
an infinite loop and unable to handle new connections, even though the server
process is still running.

When a container has a healthcheck specified, it has a _health status_ in
addition to its normal status. This status is initially `starting`. Whenever a
health check passes, it becomes `healthy` (whatever state it was previously in).
After a certain number of consecutive failures, it becomes `unhealthy`.

The options that can appear before `CMD` are:

* `--interval=DURATION` (default: `30s`)
* `--timeout=DURATION` (default: `30s`)
* `--retries=N` (default: `1`)

The health check will first run **interval** seconds after the container is
started, and then again **interval** seconds after each previous check completes.

If a single run of the check takes longer than **timeout** seconds then the check
is considered to have failed.

It takes **retries** consecutive failures of the health check for the container
to be considered `unhealthy`.

There can only be one `HEALTHCHECK` instruction in a Dockerfile. If you list
more than one then only the last `HEALTHCHECK` will take effect.

The command after the `CMD` keyword can be either a shell command (e.g. `HEALTHCHECK
CMD /bin/check-running`) or an _exec_ array (as with other Dockerfile commands;
see e.g. `ENTRYPOINT` for details).

The command's exit status indicates the health status of the container.
The possible values are:

- 0: success - the container is healthy and ready for use
- 1: unhealthy - the container is not working correctly
- 2: starting - the container is not ready for use yet, but is working correctly

If the probe returns 2 ("starting") when the container has already moved out of the
"starting" state then it is treated as "unhealthy" instead.

For example, to check every five minutes or so that a web-server is able to
serve the site's main page within three seconds:

    HEALTHCHECK --interval=5m --timeout=3s \
      CMD curl -f http://localhost/ || exit 1

To help debug failing probes, any output text (UTF-8 encoded) that the command writes
on stdout or stderr will be stored in the health status and can be queried with
`docker inspect`. Such output should be kept short (only the first 4096 bytes
are stored currently).

When the health status of a container changes, a `health_status` event is
generated with the new status. The health status is also displayed in the
`docker ps` output.

Signed-off-by: Thomas Leonard <thomas.leonard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

rewritten from github.com/moby/moby b6c7becbfe1d76b1250f6d8e991e645e13808a9c
2018-06-01 20:10:12 -07:00
John Howard 10e1cb2dcc Support platform file paths through escape
Signed-off-by: John Howard <jhoward@microsoft.com>

rewritten from github.com/moby/moby e8e3dd32c5bad727010ec787f484b98942977531
2018-06-01 20:10:12 -07:00
Yong Tang e69d12f9cc Labels set on the command line always override labels in Dockerfile
This fix tries to address the inconsistency in #22036 where labels
set on the command line will not override labels specified in
Dockerfile, but will override labels inherited from `FROM` images.

The fix add a LABEL with command line options at the end of the
processed Dockerfile so that command line options labels always
override the LABEL in Dockerfiles (or through `FROM`).

An integration test has been added for test cases specified in #22036.

This fix fixes #22036.

NOTE: Some changes are from #22266 (@tiborvass).

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

rewritten from github.com/moby/moby 5844736c14b29860ea03b040e9a052e59ad75bfc
2018-06-01 20:10:12 -07:00
Jessica Frazelle f890a7d8de fix variables that werent being called
Signed-off-by: Jessica Frazelle <acidburn@docker.com>

rewritten from github.com/moby/moby 0e025b4bb16c0d4cc6b3f0c040713d061b9b051a
2018-06-01 20:10:12 -07:00
Stefan Weil 84853bd3f3 Fix some typos in comments and strings
Most of them were found and fixed by codespell.

Signed-off-by: Stefan Weil <sw@weilnetz.de>

rewritten from github.com/moby/moby 2eee613326fb59fd168849618d14a9054a40f9f5
2018-06-01 20:10:12 -07:00
Justas Brazauskas ec19aea8e6 Fix typos found across repository
Signed-off-by: Justas Brazauskas <brazauskasjustas@gmail.com>

rewritten from github.com/moby/moby 927b334ebfc786276a039e45ec097e71bf9a104c
2018-06-01 20:10:12 -07:00
perhapszzy@sina.com e6f5e7db20 Add line information in the parsed Dockerfile
Signed-off-by: perhapszzy@sina.com <perhapszzy@sina.com>

rewritten from github.com/moby/moby 5b6c4dd3df4d501018268d5c32bf78380923c292
2018-06-01 20:10:12 -07:00
David Calavera f89b05f3f8 Remove LXC support.
The LXC driver was deprecated in Docker 1.8.
Following the deprecation rules, we can remove a deprecated feature
after two major releases. LXC won't be supported anymore starting on Docker 1.10.

Signed-off-by: David Calavera <david.calavera@gmail.com>

rewritten from github.com/moby/moby 3b5fac462d21ca164b3778647420016315289034
2018-06-01 20:10:12 -07:00
John Howard 4b2990c545 Windows: Fix dockerfile\parser unit test
Signed-off-by: John Howard <jhoward@microsoft.com>

rewritten from github.com/moby/moby 864e892e2a310c5b67dfd842e39634dd0d99da73
2018-06-01 20:10:12 -07:00
Alexander Morozov a32ceafce2 Fix dumper program to use proper import
Signed-off-by: Alexander Morozov <lk4d4@docker.com>

rewritten from github.com/moby/moby 1477a3d7fda5e8b29d68ffd5c4d6edead9ba2c61
2018-06-01 20:10:12 -07:00
Tibor Vass b4f7a7670d Move builder files to builder/dockerfile
Signed-off-by: Tibor Vass <tibor@docker.com>

rewritten from github.com/moby/moby f41230b93a77ca7f4d84718f39a4847b6117f694
2018-06-01 20:10:12 -07:00
Tonis Tiigi 6b25a56906 dockerfile: support building from a Dockerfile URL
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-30 13:20:27 -07:00
Tonis Tiigi 1067c24df2 git: support all git transports
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-29 10:56:08 -07:00
Tibor Vass 9adfee1475 dockerfile: use updated tonistiigi/copy:v0.1.1 image
https://github.com/tonistiigi/copy/pull/3

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-05-24 20:57:45 +00:00
Tõnis Tiigi 6a103ab0ac
Merge pull request #403 from tonistiigi/modtime
Fixes for cache invalidation on modtime change
2018-05-24 13:51:50 -07:00
Tonis Tiigi 5fbaff4f66 dockerfile: add test for cmd override and shell
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-23 13:27:35 -07:00
Tonis Tiigi 1fad598adf dockerfile: add withShell
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-23 12:19:10 -07:00
Tonis Tiigi 50386f3772 dockerfile: reset cmd from parent on entrypoint
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-23 12:18:14 -07:00
Tonis Tiigi eefc49ff72 dockerfile: detect empty images early
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-22 16:54:04 -07:00
Tonis Tiigi 8084e8cde9 llbsolver: allow empty returns for opts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-22 15:46:32 -07:00
Tonis Tiigi 53738b8211 dockerfile: allow ignorecache on copy
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-22 10:50:36 -07:00
Akihiro Suda 779d8fa5b2
Merge pull request #400 from tonistiigi/proxy-args
Add proxy args support
2018-05-22 16:04:12 +09:00
Tonis Tiigi a9dea8b235 dockerfile: add global arg test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-21 17:10:21 -07:00
Tonis Tiigi 286cf44db3 dockerfile: add default build-arg support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-21 16:08:12 -07:00
Akihiro Suda 199eb01177 dockerfile: fix `ADD https://.../foo.tar.gz /local/foo.tar.gz`
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-05-21 15:15:27 +09:00
Tonis Tiigi 1e096c4ce9 dockerfile: allow builds from http context
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-16 23:03:52 -07:00
Akihiro Suda 10003f2c2d
Merge pull request #384 from tonistiigi/syntax-directive
dockerfile: add syntax directive for introducing new features
2018-05-17 11:16:47 +09:00
Akihiro Suda 80f113196d
Merge pull request #390 from tiborvass/fix-nil-labels-panic
dockerfile: ensure Labels map is not accessed when nil
2018-05-17 11:15:10 +09:00
Akihiro Suda c4bffb7e06
Merge pull request #382 from moby/llbsolver-next
Upgrade to llbsolver-next
2018-05-17 11:14:23 +09:00
Tibor Vass a2d2551dcd dockerfile: ensure Labels map is not accessed when nil
Signed-off-by: Tibor Vass <tibor@docker.com>
2018-05-17 01:30:19 +00:00
Tibor Vass ae37b85b24 dockerfile: fix hanging when encountering invalid dockerignore pattern
This commit fixes the case when "!" is provided alone as a dockerignore
pattern resulting in buildkit hanging. An integration test is added to
guard the bug.

The bug was due to incorrect error propagation in the fsutil package.
Thus this commit vendors a newer, fixed version of fsutil.

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-05-16 23:26:43 +00:00
Tonis Tiigi 460fb33da7 dockerfile: add syntax directive for introducing new features
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-15 10:00:48 -07:00
Tonis Tiigi 71f5e6dd21 solver: rename solver-next to solver
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi a55d206cdb dockerfile: add no-cache support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi a7bc9b9fd2 cache: maintain creation time with remote cache
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi cea5d60dfa dockerfile: add cache import/export test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi 644540e0f1 gateway: update gateway to cache import refs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi 966d3f7b32 dockerfile: add cache-from support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi 11ada4c27b frontend: enable importref for frontends
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi 3ca07f1467 api: allow multiple cache import refs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Tonis Tiigi b6c782d53e control: switch to use solver-next
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
Akihiro Suda 72c08b5cb9 update containerd (binary: v1.1.0, library: May 11, 2018)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-05-11 15:46:15 +09:00
Tonis Tiigi b80f45955a solver: allow exporters to set data to solve results
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-07 20:52:11 -07:00
Tonis Tiigi d51a921ab4 dockerfile: add test making sure cache generates same id
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-04 15:51:26 -07:00
Tonis Tiigi b8b55a8c22 image: export reproducible timestamps
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-04 15:51:19 -07:00
Tonis Tiigi f9d1e4c32c dockerfile: add test for labels
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-03 16:24:26 -07:00
Tonis Tiigi e24e194078 remove net/context references
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-30 18:10:54 -07:00
Tonis Tiigi e5e8755dff dockerfile: add tests for case insensitive targets
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-27 10:08:27 -07:00
Tonis Tiigi 864adf02dd dockerfile: fix context path detection for var substitution
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-23 11:37:53 -07:00
Tonis Tiigi 8b85812dda dockerfile: allow skipping resolving manifest
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-19 21:39:58 -07:00
Tonis Tiigi 0b203b21df dockerfile: copy only files that are used
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-13 21:13:29 -07:00
Tõnis Tiigi 89f8bc1c02
Merge pull request #329 from AkihiroSuda/containerd-20180403
update containerd (binary: v1.0.3, library: Apr 3, 2018)
2018-04-04 10:05:42 -07:00
Akihiro Suda 126f91dea7 update containerd (binary: v1.0.3, library: Apr 3, 2018)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-04-04 17:22:20 +09:00
Akihiro Suda aabfa3b84a
Merge pull request #324 from ijc/readonly-rootfs-execop
Make llb.ReadonlyRootFS usable with common container images
2018-04-04 11:43:44 +09:00
Ian Campbell d9b64588cb Regenerate protobuf go file with gogoproto v0.5
This matches the version in vendor.conf, the previous version was `v0.5~25`
according to `git describe`.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 13:45:50 +01:00
Ian Campbell ddae5a6ea4 Make llb.ReadonlyRootFS usable with common container images
e.g. with busybox image:

    OCI runtime create failed: container_linux.go:348:
    starting container process caused "process_linux.go:402:
    container init caused \"rootfs_linux.go:58:
    mounting \\\"proc\\\" to rootfs \\\"/.../rootfs\\\" at \\\"/proc\\\"
    caused \\\"mkdir /.../rootfs/proc: read-only file system\\\"\"": unknown

This is because we were setting the underlying snapshot readonly so the various
mountpoints (here /proc) cannot be created. This would not be necessary if
those mountpoints were present in images but they typically are not.

The right way to get around this (used e.g. by `ctr`) is to use a writeable
snapshot but to set root readonly in the OCI spec. In this configuration the
rootfs is writeable when mounts are processed but is then made readonly by the
runtime (runc) just before entering the user specified binary within the
container.

This involved a surprising amount of plumbing.

Use this new found ability in the dockerfile converter's `dispatchCopy`
function.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 11:04:07 +01:00
Tõnis Tiigi 44b843d50d
Merge pull request #323 from ijc/protobuf-regen
Validation and (controlled) generation for go generate'd files.
2018-03-28 10:12:14 -07:00
Ian Campbell 17069fe7e4 Validation and (controlled) generation for go generate'd files.
Modelled after the vendor support provide a validator and an updator for files
produced by `go generate` (which today just means `*.pb.go`).

Main difference from the vendor support is that we are no longer simply nuking
and replacing a single directory, so I ended up hardcoding `*.pb.go` in a bunch
of places which I don't like but cannot see a way around which doesn't risk
nuking people's other local changes.

The generated files are placed in an unpacked form in a `FROM scratch`
container for update. Use a subdirectory and `tar --strip-components` (portable
to MacOS and Linux according to `tar(1)`) since trying to do a `docker export`
of just the root ends up adding `.dockerenv`, `sys`, `proc` and `dev` to the
source tree.

The validate container is not `FROM scratch` because we want `cat`.

The run in `frontend/gateway/pb/generate.go` was missing an include so fix
that.

The versions of `protoc` and the gogo plugins were chosen to regenerate the
existing code as closely as possible. The updates to `*.pg.go` here are all the
result of regenerating with go1.9 which fixed
https://github.com/golang/go/issues/17663 and replaced an invalid timestamp in
the gzip header of the data encoded in `fileDescriptor*`, and adopted a new
standard for marking generated files.

Finally, I noticed that my `docker run`s were missing an `--rm` which I
inherited from `validate-vendor`, so fix all those.

Closes: #322

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-28 10:52:16 +01:00
Akihiro Suda 9ef8233da1 client: support passing io.WriteCloser via SolveOpt for FSSyncTargetFile
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-03-28 15:48:54 +09:00
Tonis Tiigi 12fd7edc24 gateway: typo fix
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-08 10:13:26 -08:00
Tonis Tiigi cd110f6b10 dockerfile: fix copy from implicit stage
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-21 18:17:30 -08:00
Tonis Tiigi 7906fb3085 vendor: update containerd to v1.0.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-13 16:00:09 -08:00
Daniel Nephin f6e9a96636 Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-05 16:51:57 -05:00
Tonis Tiigi ad6ab8aba2 dockerfile: new tests for copy
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-05 14:33:48 -05:00
Akihiro Suda 994c1a63af
Merge pull request #275 from vdemeester/add-linter-deadcode
Add deadcode to the linter
2018-02-02 11:13:29 +09:00
Vincent Demeester 79e346f59a
Add deadcode to the linter
it tends to help keeping uneeded code out O:)

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-02-01 17:59:04 -08:00
Vincent Demeester 2f162c1138
Remove shell parser for dockerfile2llb, using moby upstream package
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-02-01 17:46:51 -08:00
Vincent Demeester 08373f2381
Clean some small dependencies on docker/docker package
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-02-01 16:05:24 -08:00
Tonis Tiigi 6e40e83d35 Remove net/context dependencies
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-17 09:11:22 -08:00
Tonis Tiigi 29b72a3912 solver: add some trace points
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-08 18:03:12 -08:00
Tonis Tiigi 6cb4256e81 dockerfile: fix copy with wildcards and add tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-26 15:41:09 -08:00
Tonis Tiigi 3407de2e04 gateway: allow skipping tag in source parameter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-20 22:51:33 -08:00
Tonis Tiigi 7f513a4fd4 dockerfile: add copy —-chown support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-19 20:54:26 -08:00
Akihiro Suda 2a33b2e822 *: update build tags (`containerd` -> `no_oci_worker`)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-18 15:47:56 +09:00
Tonis Tiigi 367752bf1c dockerfile: add test for building from git repo
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-17 21:09:20 -08:00