JohnHammond
/
buildkit
mirror of https://github.com/JohnHammond/buildkit.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,511 Commits
7 Branches
72 Tags
27 MiB
Commit Graph

1511 Commits (05766c5c21a1e528eeb1c3522b2f05493fe9ac47)

Author SHA1 Message Date
Tonis Tiigi 8360d73946 testutil: avoid overriding cleanup to nil on error 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-10-05 17:21:34 -07:00
Tibor Vass c9b01072d0 solver: specify SSH key ID in error message when required key was not forwarded 
Signed-off-by: Tibor Vass <tibor@docker.com>
 2018-10-05 22:50:07 +00:00
Akihiro Suda 98e26e9685
Merge pull request #668 from tonistiigi/ssh-required-opt 
dockerfile: fix ssh required option
 2018-10-06 06:50:27 +09:00
Tonis Tiigi 48ec136f1b dockerfile: update default copy image 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-10-05 14:35:33 -07:00
Tõnis Tiigi 914e64243d
Merge pull request #667 from ijc/readdir-cap-check 
gateway: check for `ReadDir` and `StatFile` caps on client side
 2018-10-05 14:20:10 -07:00
Tonis Tiigi 079b429c12 dockerfile: fix ssh required option 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-10-05 14:10:09 -07:00
Ian Campbell 2ea0269ec4 gateway: check for `ReadDir` and `StatFile` caps on client side 
This will produce a "requested feature is not supported" style error message
with some more specific details about the build server configuration rather
than a more generic gRPC "unsupported" error message.

Ensure that the tests pass the caps to `Marshal` so they see the new
functionality.

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-10-05 15:20:50 +01:00
Tõnis Tiigi 520201006c
Merge pull request #665 from tiborvass/fix-dockerfile-load-output 
dockerfile: change output when loading Dockerfile
 2018-10-04 19:22:00 -07:00
Tõnis Tiigi ad7e104d02
Merge pull request #642 from ijc/readdir 
Add `ReadDir` to the gateway interface
 2018-10-04 14:10:38 -07:00
Tibor Vass ef99a677c5 dockerfile: change output when loading Dockerfile 
Previously, the output said "load Dockerfile from " + filename when the
file is not named Dockerfile. With this patch, it is changed to always
show "load build definition from " + filename.

This is because `# syntax = ...` can be put in non-Dockerfiles.

Signed-off-by: Tibor Vass <tibor@docker.com>
 2018-10-04 20:52:15 +00:00
Tibor Vass f07efb78e3
Merge pull request #664 from tonistiigi/clean-config 
executor: clean up static config files
 2018-10-03 15:40:33 -07:00
Tonis Tiigi 76692bbe5f executor: clean up static config files 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-10-03 13:59:33 -07:00
Tibor Vass a0c740485c
Merge pull request #663 from tonistiigi/opentracing-fix 
solver: fix opentracing providers
 2018-10-03 11:39:29 -07:00
Tõnis Tiigi 609c257d62
Merge pull request #654 from tiborvass/fix-flightcontrol-stack-overflow 
flightcontrol: add exponential backoff to Group.Do
 2018-10-03 11:35:44 -07:00
Tibor Vass 9c6fd10eae
Merge pull request #660 from tonistiigi/spec-fixes 
executor: fix spec incompatibilities
 2018-10-03 11:34:41 -07:00
Tibor Vass ae84fa2c9a flightcontrol: add exponential backoff to Group.Do 
This patch fixes a stack overflow panic by calling Do in a loop
instead of relying on recursion, with an added exponential
backoff which errors out if greater than 3 seconds.

Signed-off-by: Tibor Vass <tibor@docker.com>
 2018-10-03 17:56:18 +00:00
Ian Campbell b4454a169f implement `gateway.Reference.StatFile` method 
Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-10-03 12:16:45 +01:00
Ian Campbell df76f6d380 Move cache/fsutil into a separate cache/util subpackage 
Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-10-03 12:16:44 +01:00
Ian Campbell b30ee91a53 implement `gateway.Reference.ReadDir` method 
The alternative is a helper container running e.g. find, capturing the output
to a scratch to be read back (with ref.ReadFile) and parsed.

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-10-03 12:16:44 +01:00
Ian Campbell 531ee8d4ea cache: refactor snapshot management out of ReadFile 
Allowing for other operations within a mount.

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-10-03 12:16:44 +01:00
Ian Campbell 8b69292087 Bump github.com/tonistiigi/fsutil dependency to f567071bed24 
7e391b0e78...f567071bed

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-10-03 11:46:35 +01:00
Tonis Tiigi 9dc04755dd executor: reset file limit in default spec 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-10-02 23:24:43 -07:00
Tonis Tiigi cd451ec063 executor: make sure hostname is resolvable 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-10-02 23:24:37 -07:00
Tonis Tiigi ec0e352aae executor: mount cgroups in default spec 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-10-02 23:22:07 -07:00
Tõnis Tiigi c9dc8352a4
Merge pull request #656 from AkihiroSuda/cap-ssh 
define CapMountSSH
 2018-10-02 23:21:17 -07:00
Tõnis Tiigi 083d8b6ad9
Merge pull request #662 from AkihiroSuda/update-rootlesskit 
update rootlesskit
 2018-10-02 23:18:07 -07:00
Tõnis Tiigi 8b920d1dbf
Merge pull request #655 from AkihiroSuda/df-ssh 
dockerfile: add support for RUN --mount=type=ssh
 2018-10-02 23:16:01 -07:00
Tonis Tiigi b26e666db9 solver: fix opentracing providers 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-10-02 23:11:17 -07:00
Akihiro Suda 758a267235 s/CapMount/CapExecMount/g 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-10-03 15:03:36 +09:00
Akihiro Suda f542300a10 client/llb: fix cap marshalling 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-10-03 15:02:08 +09:00
Akihiro Suda 900a1b96c4 define CapMountSSH 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-10-03 15:00:10 +09:00
Akihiro Suda 40c8b4828a update rootlesskit 
For fix subgid interpretation (rootless-containers/rootlesskit@7c48b83f9c)

Full changes: https://github.com/rootless-containers/rootlesskit/compare/20b0fc...d843aa

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-10-03 14:57:25 +09:00
Akihiro Suda fa92effc4b dockerfile: add support for RUN --mount=type=ssh 
* Needs to be compiled with `dfrunmount dfssh`
* Implemented options:
 * `type`(required): needs to be `ssh`
 * `target`(optional): the socket path in the container
 * `id`(optional): id

Test script:

	#!/bin/bash
	set -exu -o pipefail
	REF=localhost:5000/dfssh:latest
	ssh-add -l
	sudo buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=frontend/dockerfile/cmd/dockerfile-frontend \
	  --frontend-opt "build-arg:BUILDTAGS=dfrunmount dfssh" \
	  --exporter=image --exporter-opt name=$REF --exporter-opt push=true
	mkdir -p /tmp/foo
	cd /tmp/foo
	cat << EOF > Dockerfile
	# syntax=$REF
	FROM alpine
	RUN apk add --no-cache openssh-client
	RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan gitlab.com >> ~/.ssh/known_hosts
	RUN --mount=type=ssh ssh git@gitlab.com
	# "Welcome to GitLab, @GITLAB_USERNAME_ASSOCIATED_WITH_SSHKEY" should be printed here
	EOF
	sudo buildctl build --ssh default=$SSH_AUTH_SOCK --progress=plain --frontend=dockerfile.v0 --local context=. --local dockerfile=.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-10-03 14:53:43 +09:00
Tibor Vass b9859b2d49 dockerfile: RUN's CustomName no longer consume quotes and only replaces environment variables if set 
```
FROM alpine
RUN echo "Hey $name"
```

When building the above Dockerfile, the RUN command used to show: `echo Hey `
With this patch it now shows `echo "Hey $name"`.

However, because it is very useful for debugging to see the values variables can take,
they are expanded if set.

```
FROM alpine
ENV name=Jude
RUN echo "Hey $name"
```

Building the above Dockerfile makes the RUN command show: `echo "Hey Jude"`.

Signed-off-by: Tibor Vass <tibor@docker.com>
 2018-10-03 02:41:51 +00:00
Akihiro Suda cc8d59ecb0
Merge pull request #659 from tonistiigi/fix-dfextall 
dockerfile: fix building dfextall
 2018-10-03 10:31:39 +09:00
Tonis Tiigi d40d557692 dockerfile: fix building dfextall 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-10-02 10:33:18 -07:00
Tõnis Tiigi 259d39ff42
Merge pull request #652 from tiborvass/fix-build-arg-run-output 
dockerfile: substitute build args in addition to env vars in RUN's CustomName
 2018-10-02 09:48:01 -07:00
Tõnis Tiigi f14886ac1d
Merge pull request #650 from tonistiigi/solver-test 
solver: add TestSlowCacheAvoidLoadOnCache test
 2018-10-01 10:57:11 -07:00
Tibor Vass 38198bd5c3 dockerfile: substitute build args in addition to env vars in RUN's CustomName 
Example Dockerfile:

 FROM busybox
 ARG foo=abc
 ENV bar=def
 RUN echo $foo $bar $baz

Output without fix:

 #5 [2/2] RUN echo  def
 #5       digest: sha256:647df948c9689163efaf92b24b38779bc9f8b350482ecc3de3533f86a544c196
 #5         name: "[2/2] RUN echo  def "
 #5      started: 2018-09-29 19:01:11.376665368 +0000 UTC
 #5 0.764 abc def
 #5    completed: 2018-09-29 19:01:12.320141054 +0000 UTC
 #5     duration: 943.475686ms

Output with fix:

 #5 [2/2] RUN echo abc def
 #5       digest: sha256:647df948c9689163efaf92b24b38779bc9f8b350482ecc3de3533f86a544c196
 #5         name: "[2/2] RUN echo abc def "
 #5      started: 2018-09-29 19:01:11.376665368 +0000 UTC
 #5 0.764 abc def
 #5    completed: 2018-09-29 19:01:12.320141054 +0000 UTC
 #5     duration: 943.475686ms

Signed-off-by: Tibor Vass <tibor@docker.com>
 2018-09-29 19:07:01 +00:00
Tõnis Tiigi 3bc8b24fe0
Merge pull request #637 from ijc/llb-image-default-path 
Push setting of default PATH down into the executor(s)
 2018-09-28 08:37:19 -07:00
Ian Campbell 720d8327e2 Push setting of default PATH down into the executor(s) 
Setting the default `PATH` in the `llb.State` on the client side means it
depends on the `GOOS` of the buildkit client, rather than of the environment
where it will actually execute.

Instead defer this to execution time and insert the default PATH at that point
if one is not present. Doing this in solver/llbsolver/ops/exec covers all
executors and also avoids breaking the cache.

Client compatibility is handled via a new capability.

Fixes #604

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-09-28 14:16:47 +01:00
Ian Campbell 6e773ab708 Add `SetDefault` to `EnvList` 
This sets an envvar only if it is not already set.

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-09-28 13:42:35 +01:00
Tibor Vass e9e9503848
Merge pull request #651 from tonistiigi/concurrent-map-fix 
imagerefchecker: fix concurrent map access
 2018-09-27 17:41:27 -07:00
Tonis Tiigi e210ffc72e imagerefchecker: fix concurrent map access 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-09-27 17:12:35 -07:00
Tonis Tiigi 3a1f59f85f solver: add TestSlowCacheAvoidLoadOnCache test 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-09-27 15:03:56 -07:00
Tibor Vass 8f4dff0d16
Merge pull request #648 from tonistiigi/solver-early 
solver: fix early resolution of cache-fast deps
 2018-09-27 14:07:21 -07:00
Tibor Vass 7142c1371e
Merge pull request #647 from tonistiigi/cancel-error 
runcexecutor: make sure to return canceled
 2018-09-27 14:01:50 -07:00
Tonis Tiigi a711fbb8f3 solver: fix early resolution of cache-fast deps 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-09-26 18:25:33 -07:00
Tonis Tiigi f99352fee1 solver: make sure to return proper canceled errors 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-09-26 17:38:16 -07:00
Tibor Vass 66fcbd1258
Merge pull request #645 from tonistiigi/update-copy 
dockerfile: add test for chown destination
 2018-09-26 10:47:52 -07:00