37bfc6546c
exporter: allow push by digest in image exporter
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-18 16:00:48 -07:00
0d17ac323e
fileop: updates with new fsutil copy pkg
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:58 -07:00
8a4674bab4
fileop: add dockerfile support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:50 -07:00
81a5fa5a2e
llbsolver: fileop implementation
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 16:22:35 -07:00
a443cfff05
fileop: resolve review comments
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 16:22:35 -07:00
89e6614b38
solver: change uid to uint
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 16:22:35 -07:00
431d11dda3
llb: add timestamp override to fileop
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 16:22:35 -07:00
5b4841f308
llb: initial fileop implementation
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 16:22:01 -07:00
f4820681ce
client: fix panic on exporting cache
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-12 11:19:29 -07:00
5c9f7b8ff0
buildctl: new CLI ("Option C+")
...
See https://github.com/moby/buildkit/pull/807#issuecomment-468146089
Close #774
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-03-06 13:20:21 +09:00
6e363c2309
client: fix strings.Replace call args
...
strings.Replace call with n=0 argument makes no sense
as it will do nothing. Probably -1 is intended.
Signed-off-by: Iskander Sharipov <quasilyte@gmail.com>
2019-03-02 00:34:48 +03:00
46bde0d6b1
test: make sure `--export-cache type=inline` is not needed for import
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-02-23 21:01:27 +09:00
3ba3f5b1ff
Merge pull request #777 from tonistiigi/export-cache-inline
...
allow exporting cache metadata in the image config
2019-01-27 06:40:59 +01:00
c0371c908f
client: avoid string formatting on custom vertex names
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-01-22 15:35:39 -08:00
5434a42a6a
client: add inline remote cache tests
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-01-21 19:39:15 -08:00
6e7617e889
support local cache exporter and importer
...
Export:
$ buildctl build ... --export-cache type=local,store=/path/to/output-dir
Import:
$ buildctl build ... --import-cache type=local,store=/path/to/input-dir
Impact on CLI:
* Old (deprecated but still effective): `--export-cache localhost:5000/myrepo:buildcache --export-cache-opt mode=max`
* New: `--export-cache type=registry,ref=localhost:5000/myrepo:buildcache,mode=max`
Impact on API:
* New fields are added to control.proto and gateway.proto. The daemon
internally translates old API calls to the new ones.
* While new API can be used for `registry` caches, the client continues
to use the legacy API for `registry` caches to ensure compatibility with
old daemons.
* To import `local` caches with a frontend, the frontend needs to support
a new frontend opt `cache-imports`.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-01-19 02:38:26 +09:00
f9a436599a
client/llb: factor out `State.WithImageConfig` from `Image`.
...
Currently if a caller wants an `Image` (with config applied) and the
corresponding config (e.g. to use a base image and tailor the config to return
as `exptypes.ExporterImageConfigKey` in the build result) then they have to
`ResolveImageConfig` and then `llb.Image(..., llb.WithMetaResolver)`, which
ends up resolving the config twice.
Pulling this code out allows them to instead to the `config := ResolveImageConfig()`
once and to then do `llb.Image(...).WithImageConfig(config)`.
This is only really necessary if you want to run things in the image context
(and thus want the config applied) as part of tailoring the config before
returning the image and config. If you aren't running things on that `Image` as
part of the build then it doesn't matter if the config isn't applied to the
`Image` and `ResolveImageConfig` + `Image(...)` (without a metaresolver) is
fine.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-10-18 10:06:00 +01:00
1f51f5ea11
testutil: avoid excessive daemons in integration tests
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-11 16:47:35 -07:00
c9b01072d0
solver: specify SSH key ID in error message when required key was not forwarded
...
Signed-off-by: Tibor Vass <tibor@docker.com>
2018-10-05 22:50:07 +00:00
ad7e104d02
Merge pull request #642 from ijc/readdir
...
Add `ReadDir` to the gateway interface
2018-10-04 14:10:38 -07:00
b4454a169f
implement `gateway.Reference.StatFile` method
...
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-10-03 12:16:45 +01:00
b30ee91a53
implement `gateway.Reference.ReadDir` method
...
The alternative is a helper container running e.g. find, capturing the output
to a scratch to be read back (with ref.ReadFile) and parsed.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-10-03 12:16:44 +01:00
8b69292087
Bump github.com/tonistiigi/fsutil dependency to f567071bed24
...
7e391b0e78...f567071bed
2018-10-03 11:46:35 +01:00
cd451ec063
executor: make sure hostname is resolvable
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-02 23:24:37 -07:00
c9dc8352a4
Merge pull request #656 from AkihiroSuda/cap-ssh
...
define CapMountSSH
2018-10-02 23:21:17 -07:00
758a267235
s/CapMount/CapExecMount/g
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-03 15:03:36 +09:00
f542300a10
client/llb: fix cap marshalling
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-03 15:02:08 +09:00
900a1b96c4
define CapMountSSH
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-03 15:00:10 +09:00
fa92effc4b
dockerfile: add support for RUN --mount=type=ssh
...
* Needs to be compiled with `dfrunmount dfssh`
* Implemented options:
* `type`(required): needs to be `ssh`
* `target`(optional): the socket path in the container
* `id`(optional): id
Test script:
#!/bin/bash
set -exu -o pipefail
REF=localhost:5000/dfssh:latest
ssh-add -l
sudo buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=frontend/dockerfile/cmd/dockerfile-frontend \
--frontend-opt "build-arg:BUILDTAGS=dfrunmount dfssh" \
--exporter=image --exporter-opt name=$REF --exporter-opt push=true
mkdir -p /tmp/foo
cd /tmp/foo
cat << EOF > Dockerfile
# syntax=$REF
FROM alpine
RUN apk add --no-cache openssh-client
RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan gitlab.com >> ~/.ssh/known_hosts
RUN --mount=type=ssh ssh git@gitlab.com
# "Welcome to GitLab, @GITLAB_USERNAME_ASSOCIATED_WITH_SSHKEY" should be printed here
EOF
sudo buildctl build --ssh default=$SSH_AUTH_SOCK --progress=plain --frontend=dockerfile.v0 --local context=. --local dockerfile=.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-03 14:53:43 +09:00
720d8327e2
Push setting of default PATH down into the executor(s)
...
Setting the default `PATH` in the `llb.State` on the client side means it
depends on the `GOOS` of the buildkit client, rather than of the environment
where it will actually execute.
Instead defer this to execution time and insert the default PATH at that point
if one is not present. Doing this in solver/llbsolver/ops/exec covers all
executors and also avoids breaking the cache.
Client compatibility is handled via a new capability.
Fixes #604
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-09-28 14:16:47 +01:00
6e773ab708
Add `SetDefault` to `EnvList`
...
This sets an envvar only if it is not already set.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-09-28 13:42:35 +01:00
f6e104da5f
gateway: allow access to apicaps
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-21 15:33:41 -07:00
39404586a5
Merge pull request #622 from tonistiigi/df-modes-switch
...
dockerfile: enable frontend selection in integration tests
2018-09-20 14:44:11 -07:00
a75983ddb9
integration: pass mirrored images from test
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-18 22:07:38 -07:00
0940cdc6fe
update golint comments
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-18 22:06:47 -07:00
36be95ea0e
Merge pull request #623 from tonistiigi/failfast
...
buildctl: replace withblock with dial error check
2018-09-17 11:44:57 -07:00
bf571a519e
update Go to 1.11
...
For consistency with Moby (moby/moby#37358 )
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-09-15 14:25:11 +09:00
653fb12ecf
buildctl: replace withblock with dial error check
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-13 17:29:01 -07:00
f7120867f1
client: add regression test for closed stdin
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-11 14:26:46 -07:00
347420e77f
sshprovider: allow keys from local files
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-10 14:12:41 -07:00
9337126e6e
client: add ssh socket test
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-07 08:57:11 -07:00
6888956557
sshforward: implement ssh socket forwarding
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-07 08:57:06 -07:00
9a6e2a5a91
Merge pull request #607 from kunalkushwaha/fix-typo-1
...
function name fixed SetMarhalDefaults -> SetMarshalDefaults
2018-09-05 22:01:39 -07:00
98c1f44151
function name fixed SetMarhalDefaults -> SetMarshalDefaults
...
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-09-06 13:28:39 +09:00
079c747c4e
ontrol: support calling automatic GC
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-04 18:13:29 -07:00
05a468d460
Merge pull request #595 from tonistiigi/uid
...
client: reset user ID for synced dirs
2018-08-31 12:14:06 -07:00
2759005f6a
Merge pull request #575 from ijc/frontend-set-export-name
...
exporters: allow frontend control of the exported image name
2018-08-31 10:27:09 -07:00
03a9cb89af
exporter: Use "*" to signify that client allows frontend to set image name
...
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-31 17:34:49 +01:00
0b20bcc8f3
The OCI image exporter does not support naming, so reject such attempts.
...
Adjust the test case to not name in the OCI case.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-31 14:23:34 +01:00
aa51b40649
Include exported image name in ExporterResponse metadata
...
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-31 11:07:32 +01:00
c9711fcc26
Merge pull request #574 from ijc/return-frontend-metadata
...
Allow some Metadata to be return from frontend to the control client
2018-08-30 10:08:51 -07:00
755bf5387e
Moved some subtests into `t.Run` blocks.
...
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-30 16:28:19 +01:00
3db7441e06
clarify naming in testFrontendImageNaming
...
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-30 15:59:12 +01:00
8d114533c2
exports: require caller to request frontend-based naming, by passing name of `%s`.
...
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-30 15:59:06 +01:00
64ff38f163
client: reset user ID for synced dirs
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-28 14:22:22 -07:00
decb27618b
containerd: set timeout explicitly (60s)
...
The contained default value was shortened from 60s to 10s in containerd/containerd#2554
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-08-23 18:14:57 +09:00
fa39d134ee
exporters: allow frontend control of the exported image name
...
Returning a metadata item named "image.name" will set the name to use.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-17 13:20:36 +01:00
fdf13cf96e
Allow some Metadata to be return from frontend to the control client
...
Propagate anything in the `frontend.*` namespace from the `frontend.Result`
into the `client.SolveResponse`.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-17 12:45:21 +01:00
af46188e9b
Merge pull request #533 from ijc/client-gateway
...
access gateway API from client
2018-08-17 05:12:23 +09:00
22f632f895
allow frontends to be run directly on the control client side
...
This allows builder code to be written which can be built as either a gateway
container or in a purely client side configuration, giving implementors more
flexibility.
Now when `Solve` sees a request with neither a definition nor a frontend
specified it will make the job available via new LLBBridge endpoints on the
control socket which the client can then use. These end points require the job
id to be present in the gRPC metadata and a client side object is added to
facilitate this.
The `llbBridgeForwarder` type is now exposed as a public `interface
LLBBridgeForwarder` which satisfies the underlying gRPC server interface
(`pb.LLBBridgeServer`) as well as a new `Done()` & `Result()` pair which can be
used to wait for the client to call `Return()` (using a model similar to
`context.Context`).
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-14 11:50:08 +01:00
130f5f5ab0
solver: net host with basic entitlements support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-09 14:03:35 -07:00
47c4888ab1
Merge pull request #551 from tonistiigi/gc-prune
...
cache: add gc style pruning support
2018-08-07 11:14:46 -07:00
1a37ee3bf4
cache: add gc style pruning support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-06 21:45:39 -07:00
bf29f5bd3b
client: add test for extra hosts
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-03 13:56:13 -07:00
f7359f7093
llb: add extrahost support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-03 13:56:13 -07:00
4c44d2c6e4
dockerfile: custom names to vertexes
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-31 10:41:30 -07:00
d1ad6961e3
cache: detect refs shared with image store
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-27 14:08:50 -07:00
97fa2b527a
cache: add all option to prune
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-26 22:55:05 -07:00
489246dd28
cache: support for internal/frontend record type
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-26 22:54:53 -07:00
57b96a0ee5
cache: add record type field to usage record
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-26 22:54:39 -07:00
aa81807f1c
Merge pull request #542 from tonistiigi/resolvemode
...
llb: add resolvemode to image source
2018-07-26 14:04:27 -07:00
9f5e5c7dc7
metaresolver: refactor to option struct and resolvemode
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 18:22:15 -07:00
a3b5d22419
llb: add resolvemode to image source
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 18:20:04 -07:00
70e0b64482
client: generalize the filter passing
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 18:19:31 -07:00
0ba24f2a90
cache: implement filter for diskusage
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 17:20:59 -07:00
f719aadcba
dockerfile: add tests for multi-platform exports
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 16:01:12 -07:00
53f657a5db
client: add secret mounts test
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-19 22:02:11 -07:00
378df8cbaa
llb: add secrets support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-19 22:02:11 -07:00
d4c57aec5d
llb: add caps support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-18 17:22:30 -07:00
c9df92f4c5
imagemetaresolver: use platform in cache
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-17 18:36:32 -07:00
1456a3f165
llb: don't fill platform on static sources
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-16 15:27:03 -07:00
dca7bdcd4c
Merge pull request #495 from tonistiigi/unique-local
...
llb: set unique ID to local sources without session
2018-07-10 20:34:04 +09:00
946a12eb42
gateway: expose worker information
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-09 16:32:39 -07:00
40e270a2e6
llb: set unique ID to local sources without session
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-09 16:27:12 -07:00
3fa70c6b25
client: simplify parallel sources test
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-09 16:27:08 -07:00
0dd25df7e2
client: add test case for multiple parallel builds.
...
It appears that multiple parallel builds suffer some cross talk between their
mount/snapshots e.g. this new case currently fails because iterations see files
in `/src/` which correspond to other iterations (i.e. iteration 1 might see a
file called "test2.txt"). The `ls -i` and `stat` output also confirms that
where tests see the same filename they are also apparently seeing the same
inode.
Note that each solve uses a `SolverOpt.SharedKey` of the (unique) source
directory so there should be no clashes. Use of `llb.sharedKeyHint()` is
avoided since using a unique one of these hides the issue (AIUI uniqueness of
`sharedKeyHint` is only required with a session, not between them).
The shell fragment within the test is a bit crazy, it tries to output as much
info about what it saw vs what was expected to aid debugging.
I have also been seeing cases in my own code where `cacheContext.checksum`
fails the `not found` test at the top, which seems like either the wrong thing
is mounted or it has gone away or otherwise been mutated unexpectedly. I had
been hoping that this test case would also reproduce this, but so far it has
not.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-07-09 15:44:28 -07:00
f3b743bd13
Merge pull request #467 from dmcgowan/export-oci-media-types
...
Support creating and pushing OCI types
2018-06-28 14:46:34 -07:00
618f34a1f9
Support creating and pushing OCI types
...
Adds image and oci exporter option "oci-mediatypes"
Ensures that the images created in the content store
have the correct type which matches the manifest.
Sets the correct media type on the descriptor in push from
reading the type specified in the manifest.
Removes use of distribution manifest packages.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-06-28 11:23:35 -07:00
e618878451
dockerfile: platform support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-25 11:06:08 -07:00
f9383c48a3
source: image source platform support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-25 10:39:17 -07:00
19612b901f
Merge pull request #462 from tonistiigi/llbconstraints
...
llb: force platform in llb and allow constraints
2018-06-26 01:03:38 +09:00
242697af59
llb: add marshal roundtrip tests for platform
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-22 17:55:36 -07:00
d34e4c68b5
llb: automatically carry platform with state chain
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-22 16:49:03 -07:00
dc9de85069
llb: force platform in llb and allow constraints
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-22 16:49:03 -07:00
a1bafadfd5
worker: add platforms support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-22 11:46:46 -07:00
ccbf185006
llbsolver: allow sharing modes for cache mounts
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-19 15:57:20 -07:00
97ffd6cce8
client: take a context.Context to New().
...
This allows two things:
- The caller to set a shorter timeout than previously hardcoded 30s. In
`buildctl` reduce the timeout to 5s. Since the existing timeout has gone
callers will need to arrange to pass one themselves.
- The caller can arrange for the context to be cancelled for other reasons, use
this in `buildctl` to plumb through the Ctrl-C handling, meaning that
`buildctl` now exits almost immediately on Ctrl-C instead of after several
seconds.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-06-11 14:28:08 +01:00
30461204da
vendor: update fsutil to fix local source bug
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-10 09:22:59 -07:00
f381aaac05
Merge pull request #445 from tonistiigi/tmpfs
...
llbsolver: add support for tmpfs mounts to exec
2018-06-09 11:52:16 +09:00
596b03a8a0
llbsolver: allow tmpfs mounts
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-08 13:05:14 -07:00
6ce604740f
vendor: replace locker pkg for legal
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-08 11:00:37 -07:00
60344aa592
Merge pull request #442 from tonistiigi/runmount
...
dockerfile: add run --mount support
2018-06-09 02:35:34 +09:00
7943598281
Merge pull request #439 from AkihiroSuda/vendor-containerd-20180606
...
update containerd library
2018-06-08 10:33:10 -07:00
cf4cc2d6d0
Merge pull request #431 from tonistiigi/dockerfile-symlinks
...
dockerfile: detect source symlinks with targets
2018-06-08 21:57:53 +09:00
47a00b84fe
update containerd library
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-06-08 21:44:17 +09:00
055dcb6c09
local: allow followpaths for local source
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-07 23:23:13 -07:00
479419abf2
llbsolver: don’t error on readonly scratch mount
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-07 16:23:53 -07:00
28fc483de0
client: use grpc.DialContext and context.WithTimeout instead of grpc.WithTimeout
...
Currently, grpc.WithTimeout is deprecated.
2018-06-05 17:34:49 +09:00
1067c24df2
git: support all git transports
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-29 10:56:08 -07:00
ef1a79a1c1
Allow caller to specify platform for use with imagemetaresolver.
...
This needs to be plumbed through to `imageutil.Config` which is also used from
the `containerimage` package. Since that is used in the context of a Worker the
`platforms.Default()` (corresponding to the platform of the worker itself) is
appropriate so apply a default at that level.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-05-24 11:23:35 +01:00
06d81be140
Merge pull request #401 from tonistiigi/relative-wd
...
llb: fix relative working directory in client
2018-05-22 16:04:24 +09:00
7e232f5181
llb: fix relative working directory in client
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-21 20:46:37 -07:00
66488dc6d8
exec: support proxy settings
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-21 16:08:11 -07:00
2bd78c3c6e
llbsolver: add support for cached mount type
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-15 12:24:20 -07:00
1b4ce4324f
buildctl: add export-cache-opt flags
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
335b1c3660
llbsolver: update for new cache variables
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
3ca07f1467
api: allow multiple cache import refs
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
4ee4ebcb13
cacheimport: fix result checking methods
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
75bc38c15b
client: add basic cache import test
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
be6da00944
Merge pull request #381 from ijc/windows-tcp-dialer
...
client: support non-npipe server addresses on Windows
2018-05-14 09:09:23 -07:00
f57186d186
client: support non-npipe server addresses on Windows
...
This makes `tcp://w.x.y.z:port` work from Windows based clients (perhaps
talking to a remote Linux based buildkitd or one running in a container with
Docker for Windows and TCP port forwarding).
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-05-14 15:15:08 +01:00
b80f45955a
solver: allow exporters to set data to solve results
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-07 20:52:11 -07:00
7ac9599c8f
session: separate authprovider to package
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-17 20:36:23 -07:00
ddae5a6ea4
Make llb.ReadonlyRootFS usable with common container images
...
e.g. with busybox image:
OCI runtime create failed: container_linux.go:348:
starting container process caused "process_linux.go:402:
container init caused \"rootfs_linux.go:58:
mounting \\\"proc\\\" to rootfs \\\"/.../rootfs\\\" at \\\"/proc\\\"
caused \\\"mkdir /.../rootfs/proc: read-only file system\\\"\"": unknown
This is because we were setting the underlying snapshot readonly so the various
mountpoints (here /proc) cannot be created. This would not be necessary if
those mountpoints were present in images but they typically are not.
The right way to get around this (used e.g. by `ctr`) is to use a writeable
snapshot but to set root readonly in the OCI spec. In this configuration the
rootfs is writeable when mounts are processed but is then made readonly by the
runtime (runc) just before entering the user specified binary within the
container.
This involved a surprising amount of plumbing.
Use this new found ability in the dockerfile converter's `dispatchCopy`
function.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 11:04:07 +01:00
3dd4d12796
client: disallow empty output for exportation
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-04-02 13:22:44 +09:00
9ef8233da1
client: support passing io.WriteCloser via SolveOpt for FSSyncTargetFile
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-03-28 15:48:54 +09:00
1cf2daae92
Merge pull request #320 from tonistiigi/contentutil
...
util: add contentutil helpers
2018-03-23 10:09:15 +09:00
0702303f6c
imagemetaresolver: switch to contentutil buffer
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-22 16:19:56 -07:00
ac6598d255
Check that mounts always have a target.
...
Otherwise the daemon panics when generating the OCI spec.
For belt and braces check in the ExecOp Run function but also when generating the spec.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-22 12:05:12 +00:00
7be9ae6954
Convert ReadonlyRootFS to a RunOption
...
This seems to have missed out on an update/refactor at some point and was not
usable in its previous form without duplicating the `runOptionFunc`
scaffolding.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-20 16:58:05 +00:00
432c191e81
client: speed up llb marshal
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-19 10:25:58 -07:00
ae0293c4bf
Add an integration test for pulling schema1 images
...
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-07 15:56:22 +00:00
23ec7e0239
client: add test for duplicate whiteouts
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-13 16:00:26 -08:00
c71799dff4
client: add test for whiteout parent dirs
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-13 16:00:26 -08:00
d295a16169
client: test duplicate entries in result tarballs
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-09 11:47:07 -08:00
79e346f59a
Add deadcode to the linter
...
it tends to help keeping uneeded code out O:)
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-02-01 17:59:04 -08:00
6e40e83d35
Remove net/context dependencies
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-17 09:11:22 -08:00
00da260047
client: enable test for parent directories
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-11 17:50:15 -08:00
b94704e55c
session: fix session closing and tracing
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-08 18:03:06 -08:00
6b98dedaef
Add grpc opentracing support to the binaries
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-08 17:49:34 -08:00
1e0b890dd1
client: fix tests to wait snapshots to become releasable
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-08 11:45:53 -08:00
8cd775d108
Merge pull request #254 from tonistiigi/fix-create-time
...
exported: fix image creation time on export
2018-01-06 15:05:24 +09:00
e0d056f418
exported: fix image creation time on export
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-05 10:51:32 -08:00
b88a98bcad
client: add integration tests for prune
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-04 23:09:05 -08:00
63ce643468
cache: add prune support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-04 23:09:05 -08:00
03f717bd32
transport: add possibility to use TLS secured transport layer on server- and clientside;
2017-12-20 13:30:46 +01:00
84c0dd0b07
worker: add labels
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-20 16:28:47 +09:00
2f2b72989c
*: buildd -> buildkitd
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-19 13:23:27 +09:00
85c0f99ba0
exporter: add Docker compatible exporter
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-18 12:19:17 -08:00
2d3f36d359
source: add sharedkeyhint to local source
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-15 15:13:15 -08:00
08e1c2990c
dockerfile: add dockerignore support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-15 15:03:35 -08:00
2e9f0c42f5
client: add test for OCI exporter
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-13 09:57:33 -08:00
6edccb7913
exporter: add OCI exporter
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-13 09:57:28 -08:00
7a5390f355
worker: add support for custom user
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-12 10:26:30 -08:00
40a0581853
worker: setup own resolvconf and hosts
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-10 18:23:24 -08:00
e15cbb0b3a
Merge pull request #196 from tonistiigi/history
...
image: set up history array on exported image config
2017-12-10 02:14:09 +09:00
b74d05da9b
imageresolver: fix concurrent map writes
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-08 21:10:25 -08:00
f8e58e359c
client: add testcase for exported history array
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-08 20:00:36 -08:00
41cd660444
Merge pull request #197 from tonistiigi/test-parallel
...
set more tests to parallel
2017-12-09 12:59:38 +09:00
e4f6a5e752
set more tests to parallel
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-08 18:19:08 -08:00
f5f3b39228
client: add tests for differ and pushed data
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-08 14:37:38 -08:00
438c850a16
dockerfile: add original command metadata to llb
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-07 16:13:50 -08:00
303d4ccab9
llb: allow defining metadata on nodes
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-07 10:12:48 -08:00
68165c07f4
exporter: allow exporting config with no snapshots
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-04 23:18:09 -08:00
e0af9d1218
source: add more properties to http source
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-04 12:00:46 -08:00
90d728e194
client: add http source integration test
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-02 19:52:15 -08:00
59628fb20f
add http source
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-02 19:52:15 -08:00
2729373a03
reduce dependencies of llb package
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-30 17:29:57 -08:00
e3f85cc1ac
integration: detect test names with reflect
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-29 18:49:04 -08:00
f343fc30ec
buildctl: add sample cli test
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-29 18:34:43 -08:00
e1ced023df
update intergration tests for reuse
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-29 17:42:28 -08:00
51d1c4166f
session: expose image auth
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-20 12:56:06 +02:00
56e1f71e6b
cache: allow pushing exported cache
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-13 15:37:55 -07:00
5986fa9947
Allow to use TCP for buildkit daemon and client communication
...
The next step is to allow TLS configuration.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-10-07 22:02:15 +02:00
59910481ca
solver: make nested build use solverequest
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-05 17:51:04 -07:00
f706b13a3f
remove generateID duplicates
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-03 16:13:35 -07:00
f89ffc2ab4
llb: per-vertex metadata (e.g. IgnoreCache)
...
cache invalidation itself it not implemented yet
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-10-03 05:36:24 +00:00
f41283caed
solver: fix shared request progress and cancellation
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-27 12:45:47 -07:00
05c39612d8
dockerfile: resolve to digest ref in frontend
...
Less hacky than keeping the cached resolver.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-22 15:49:37 -07:00
fd0f7ee2a8
frontend: fix dockerfile frontend resolver
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-15 18:09:28 -07:00
b36df4738a
frontend: add more commands to dockerfile
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-15 18:09:28 -07:00
a0c9d7c690
vendor containerd v1.0.0-beta.0
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-09-09 11:13:02 +00:00
6d335c3c86
imagesource: share config resolver with frontend
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-05 13:11:27 -07:00
e1e5a8bfac
Merge pull request #106 from tonistiigi/dockerfile-frontend
...
base of dockerfile frontend
2017-09-01 15:50:21 +09:00
2d290db465
frontend: base of dockerfile frontend
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-31 23:27:25 -07:00
d1bd12cab2
llb: skip outputs of readonly mounts
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-31 10:39:40 -07:00
41ca1c3ce5
fix CI after containerd update
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-28 11:41:53 -07:00
e5563f95a6
llb: add meta resolver from image
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-21 15:41:22 -07:00
ff99df0652
llb: add cache for marshalers
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-09 18:23:37 -07:00
c5ba23d886
client: allow empty address for the system-default value
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-08-08 08:07:51 +00:00
8162e15787
llb: add support for mounting subpath or reference
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-07 18:45:50 -07:00
9f6d9a9e78
exporter: add local exporter
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-04 17:12:19 -07:00
39b9dec52f
Merge pull request #87 from tonistiigi/snapshot-metadata
...
cache: add more metadata to snapshots
2017-07-27 09:36:10 +09:00
5ecab5e142
Merge pull request #11 from dmcgowan/containerd-worker
...
Containerd worker
2017-07-26 14:36:51 -07:00
e3c91ee898
buildctl: add filter to buildctl df
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-26 11:28:26 -07:00
38ac00090c
cache: add more metadata values to snapshots
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-26 11:28:20 -07:00
9ce5053a8b
Update containerd version
...
Fix for logrus rename, use fork until fixed in moby.
Removed unused tar stream.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-26 11:07:45 -07:00
93757aef7f
nested llb builds support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-24 18:09:16 -07:00
e4b11deb3d
llb: add missing runoptions and fixing scratch input
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-24 12:34:26 -07:00
ce6a428195
rework llb package
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-23 16:53:09 -07:00
2e89d2dc72
llb: add custom types for input indexes
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-21 16:39:25 -07:00
cb168bc954
source: add multiple dir support for local source
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-20 21:17:23 -07:00
ff951eecd9
llb: add readonly mounts support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-18 16:07:37 -07:00
eac79f7c7e
cache: refactor reference reuse and caching
...
Replaces previous mutable.Freeze logic with
commits that can live together with mutable data.
Finalize method is added if the implementation
needs to make sure that the immutable ref is
flushed to the driver. Refs are automaitcally
finalized when writable layers are created on
top of them.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-18 16:01:01 -07:00
2e48745f57
session: remove confusing uuid variable
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-17 16:08:26 -07:00
5c2e675d18
source: add local file source fundamentals
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-12 23:23:20 -07:00
6ac87fd744
support building on windows (no functional worker yet)
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-07-12 06:43:30 +00:00
45bfdd5e42
exporter: add image exporter
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 23:29:14 -07:00
bf4e8c1610
llb: update runoptions signature
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 22:19:05 -07:00
2be8499bdc
llb: add scratch support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 22:19:05 -07:00
fa7ee2e2a8
source: add keepgitdir option for git source
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 22:19:05 -07:00
b65989a5e4
source: add git source
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 22:19:05 -07:00
bcb950915b
llb: add const for root mount
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 21:54:38 -07:00
25ba9d72de
solver: first pass of instruction cache
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-06 20:42:17 -07:00
6d1400a507
cache: fix reclaimable data view
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-30 22:24:32 -07:00
2df68a5706
solver: add error reporting and canceling support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-29 15:31:08 -07:00
10ab37dddb
llb: mark the functions that allow formatting
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-26 11:51:27 -07:00
991ab8d9a3
llb: add more helpers and tests to llb client
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-26 11:03:12 -07:00
b573060eea
project: remove poc label
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-23 09:37:36 -07:00
7d835ce16c
vendor: update containerd
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-22 12:14:39 -07:00
728de510f3
llb: improve llb generation
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-21 15:21:36 -07:00
5244a14bc2
client: add integration tests
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-19 14:35:57 -07:00
3d3190573c
client: save logs to trace file
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-19 14:32:51 -07:00
8e7d0904ce
client: handle status log in client
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-16 16:08:53 -07:00
a8303438d0
example: more complicated build example
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-15 16:09:03 -07:00
4b41158252
solver: control vertexstatus for pull
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-15 16:08:45 -07:00
177aac4018
api: fix types in protobuf
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-13 23:35:35 -07:00
12a84368df
solver: vertex progress reporting
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-13 23:35:29 -07:00
62b7d04d01
control: base of status reporting
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-13 22:52:59 -07:00
84a14f44ed
llb: ops execution example
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-09 14:49:01 -07:00
bf8f1ee37a
control: add simple solve method
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-09 14:49:01 -07:00
ad25e5e54c
client: implement diskusage
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-09 14:49:01 -07:00
Tonis Tiigi
Akihiro Suda
Iskander (Alex) Sharipov
Akihiro Suda
Ian Campbell
Tibor Vass
Tibor Vass
Kunal Kushwaha
Derek McGowan
Nao YONASHIRO
Vincent Demeester
Tino Rusch