e7759a861b
client: add cni network tests
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-10 18:00:06 -07:00
bc7a6f2556
add cni networking support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-10 14:42:30 -07:00
a61393a5bc
Merge pull request #1044 from tonistiigi/fromremote-gc-race
...
worker: fix gc race on fromremote
2019-06-26 15:41:27 +08:00
8e692507ee
executor: allow setting DNSConfig to be used by resolvconf
...
This patch allows downstream code to pass a DNSConfig that is
then used by executor/oci.GetResolvConf.
This would allow the BuildKit-based builder in Docker to honor
the docker daemon's DNS configuration, thus fixing a feature gap
with the legacy builder.
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-17 20:43:33 +00:00
276fee0727
worker: fix gc race on fromremote
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-06-17 10:42:08 -07:00
74190cfff8
worker: add leases to fromremote
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-05-28 14:39:05 -07:00
8e25187cc7
pull: fix content blobs deletion on pull race
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-05-28 14:39:05 -07:00
ffad8ada79
exporter: support unpack opt for image exporter
...
It is enhancement which allows to unpack image into containerd
snapshotter storage by `--output type=image,<.>=<.>,unpack=true`.
In order to support this feature, we needs to extend the Snapshotter
witwh `Name() string` function. Because we needs to set gc label for
snapshotter which need snapshotter name.
fix : #908
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-04-24 21:24:06 +08:00
894da35fed
fix unit tests for idmapping support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-04-02 20:35:18 -07:00
9f53ea3d78
userns support for sources and executor
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-02 18:26:04 -07:00
6921dbe0f6
snapshot: base for userns remapping
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-02 18:23:46 -07:00
c1a1d7033d
exporter: add tar exporter
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-27 09:40:24 -07:00
81a5fa5a2e
llbsolver: fileop implementation
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 16:22:35 -07:00
5b4841f308
llb: initial fileop implementation
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 16:22:01 -07:00
f472cb193f
exec: fix platform value in cache key
...
Use vertex platform instead of the current system one. This shouldn't break any cache unless qemu was used. Even when qemu was used the issue would not cause any cache collisions because the base images for run were scoped by platform, but better to stop using invalid data for cache key calculation.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-02-28 11:17:10 -08:00
26010a4864
decouple SessionManager from Worker
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-02-23 22:55:17 +09:00
7d41912f36
remotecache: keep layer description
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-01-21 19:39:15 -08:00
ac4037a1d9
solver: keep original cache creation date when copying between stores
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-01-18 11:19:34 -08:00
c54f4a986d
support --oci-worker-no-process-sandbox
...
Note that this mode allows build executor containers to kill (and potentially ptrace) an arbitrary process in the BuildKit host namespace.
This mode should be enabled only when the BuildKit is running in a container as an unprivileged user.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-01-08 10:42:52 +09:00
a62e7cf1e0
worker/base: allow running without `git` installed
...
genuinetools/img#195
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-11-28 18:43:43 +09:00
ddfb7f77c9
hack: update test dockerfiles to buildkit
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-10 23:26:05 -07:00
89e8097d4c
fix setting uncompressed label on content
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-08 23:11:25 -07:00
688972691f
Fix typo in error getting diff pairs
2018-09-21 13:56:05 -07:00
aa73951164
Merge pull request #631 from jhowardmsft/boltdb
...
Revendoring to move boltdb to bbolt
2018-09-19 13:59:08 +09:00
2de2c04c8e
Revendoring to move boltdb to bbolt
...
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-09-18 11:18:08 -07:00
757795e556
cache: fix usage counters
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-14 18:44:44 -07:00
907411c7e8
Merge pull request #605 from fuweid/feature_add_namespace_in_config
...
cmd,worker: support customize namespace for containerd worker
2018-09-11 11:56:24 +09:00
53aa4053bb
cmd,worker: support customize namespace for containerd worker
...
Allow user to customize containerd namespace, not just `buildkit`. It
can help user to integration with existing data easily.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2018-09-11 09:41:42 +08:00
ecd4a22546
resolver: add config support for mirrors/plainhttp
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-07 18:44:45 -07:00
a9fe50acf1
Merge pull request #580 from AntaresS/handle-cgroups
...
allow customized cgroup-parent for runcexecutor
2018-09-06 11:15:51 -07:00
b0677e7ef1
allow customized cgroup-parent for runcexecutor
...
Signed-off-by: Anda Xu <anda.xu@docker.com>
2018-09-05 12:53:57 -07:00
079c747c4e
ontrol: support calling automatic GC
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-04 18:13:29 -07:00
b97bc71adb
Refactor networking with ns paths
...
This fixes the issues where buildkit and callers do not have to be a
subpreaper in order to use networking. I can add CNI provider later,
with a hidden sub command to create a new network namespace and bind
mount it to buildkit's state dir.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-21 13:37:47 -04:00
f8dd602282
runc: improve canceling
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-07 11:51:56 -07:00
765f1b64b9
executor: allow network providers
...
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-07 11:51:56 -07:00
d1ad6961e3
cache: detect refs shared with image store
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-27 14:08:50 -07:00
aa81807f1c
Merge pull request #542 from tonistiigi/resolvemode
...
llb: add resolvemode to image source
2018-07-26 14:04:27 -07:00
87f43efda5
cache: implementation for prune filtering
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 19:35:49 -07:00
9f5e5c7dc7
metaresolver: refactor to option struct and resolvemode
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 18:22:15 -07:00
a7fedd9c9f
control: clean up cache metadata after prune
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-23 16:27:01 -07:00
2da2aff164
llbsolver: add secret mounts
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-19 22:02:05 -07:00
bc765861be
diff: implement windows layer support for linux
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-16 16:33:21 -07:00
946a12eb42
gateway: expose worker information
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-09 16:32:39 -07:00
f3b743bd13
Merge pull request #467 from dmcgowan/export-oci-media-types
...
Support creating and pushing OCI types
2018-06-28 14:46:34 -07:00
618f34a1f9
Support creating and pushing OCI types
...
Adds image and oci exporter option "oci-mediatypes"
Ensures that the images created in the content store
have the correct type which matches the manifest.
Sets the correct media type on the descriptor in push from
reading the type specified in the manifest.
Removes use of distribution manifest packages.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-06-28 11:23:35 -07:00
f9383c48a3
source: image source platform support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-25 10:39:17 -07:00
19612b901f
Merge pull request #462 from tonistiigi/llbconstraints
...
llb: force platform in llb and allow constraints
2018-06-26 01:03:38 +09:00
85e98104cf
worker: remove arch/os labels
...
Superceded by platforms array.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-24 22:49:06 -07:00
242697af59
llb: add marshal roundtrip tests for platform
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-22 17:55:36 -07:00
a1bafadfd5
worker: add platforms support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-22 11:46:46 -07:00
e991a846be
Default mounts set for buildkit
...
containerd/oci_specs mounts /run, which is not
required by buildkit.
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-06-07 14:43:44 +09:00
adef0dedef
oci-worker: experimental support for rootless mode
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-05-31 16:05:13 +09:00
8084e8cde9
llbsolver: allow empty returns for opts
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-22 15:46:32 -07:00
2bd78c3c6e
llbsolver: add support for cached mount type
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-15 12:24:20 -07:00
71f5e6dd21
solver: rename solver-next to solver
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
f00ba89a30
llbsolver: support detecting existing remotes
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
a7bc9b9fd2
cache: maintain creation time with remote cache
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
77c2793ebb
remove unused packages
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
b6c782d53e
control: switch to use solver-next
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 09:53:34 -07:00
e24e194078
remove net/context references
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-30 18:10:54 -07:00
2dba0488c8
runc: make command name configurable
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-04-17 20:39:19 -07:00
89f8bc1c02
Merge pull request #329 from AkihiroSuda/containerd-20180403
...
update containerd (binary: v1.0.3, library: Apr 3, 2018)
2018-04-04 10:05:42 -07:00
126f91dea7
update containerd (binary: v1.0.3, library: Apr 3, 2018)
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-04-04 17:22:20 +09:00
4bae684e32
Invert tags on runc worker test so they run by default.
...
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 11:01:46 +01:00
a0a7301ea0
Merge pull request #312 from tonistiigi/throttle-gc
...
worker: throttle calls to gc
2018-03-26 09:34:35 -07:00
b0f34cb8cd
export cache
...
Signed-off-by: Jess Frazelle <acidburn@microsoft.com>
2018-03-21 12:34:05 -04:00
da633a8e6b
worker: throttle calls to gc
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-03-19 10:40:31 -07:00
18280e73cc
Merge pull request #289 from ijc/allow-local-image-for-gateway-fe
...
Use local images if pull fails
2018-03-06 09:59:17 -08:00
ac5091c10b
Use local images if pull fails
...
When using `--frontend=gateway.v0` in order to use a tagged image as the
frontend it is currently required to push the tag to a registry, so that
buildkit can pull it. One can use the `image@sha256:digest` form but this is
inconvenient for the development cycle.
Instead, introduce a proxy `Resolver` which if the pull fails checks the local
image store (if there is one, which is worker dependent) for a suitable image
and if so uses that.
With this I can now:
buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=. --exporter=image --exporter-opt=name=docker.io/ijc25/fe:dev
builtctl build --frontend=gateway.v0 --frontend-opt=source=ijc25/fe:dev --exporter=image --exporter-opt=name=hello-world:dev
Without any push to a registry.
I am using the containerd worker.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-03-06 12:19:34 +00:00
de73349f01
worker/oci: allow specifying arbitrary snapshotter factory
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-02-26 18:15:10 +09:00
622d4a442a
worker/oci: allow using naive snapshotter
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-02-20 12:13:18 +09:00
7906fb3085
vendor: update containerd to v1.0.2
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-13 16:00:09 -08:00
6e40e83d35
Remove net/context dependencies
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-17 09:11:22 -08:00
b0679c66db
snapshot: clean up snapshot interface
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-04 23:09:05 -08:00
56353f99a8
snapshot: combine nogc snapshotting helpers
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-04 23:09:05 -08:00
63ce643468
cache: add prune support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-01-04 23:09:05 -08:00
84c0dd0b07
worker: add labels
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-20 16:28:47 +09:00
dca082582e
worker, solver: s/ResolveVertex/ResolveOp/g
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-19 13:20:30 +09:00
85c0f99ba0
exporter: add Docker compatible exporter
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-18 12:19:17 -08:00
21bebaea80
solver, worker: minor cleanup
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-18 16:20:59 +09:00
efde4f2340
worker, solver: update interfaces
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-18 16:20:59 +09:00
2a33b2e822
*: update build tags (`containerd` -> `no_oci_worker`)
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-18 15:47:56 +09:00
a24ab34522
buildctl: fix dump-metadata
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-15 16:44:09 +09:00
682ac0248c
Merge pull request #211 from tonistiigi/ociexport
...
exporter: add OCI exporter
2017-12-14 11:05:08 +09:00
6edccb7913
exporter: add OCI exporter
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-13 09:57:28 -08:00
8840dc7439
image: separate imagewriter to own struct
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-12 22:12:35 -08:00
89450211fa
*: deduplicate instructioncache definitions
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-13 15:03:14 +09:00
e5e24af1f5
runc: change runc test to require
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-12 14:59:24 -08:00
c3aa849014
multi-worker daemon
...
- [X] put multiples workers in a single binary ("-tags containerd standalone")
- [X] add worker selector to LLB vertex metadata
- [X] s/worker/executor/g
- [X] introduce the new "worker" concept https://github.com/moby/buildkit/pull/176#discussion_r153693928
- [X] fix up CLI
- [X] fix up tests
- allow using multiples workers (requires inter-vertex cache copier, HUGE!) --> will be separate PR
Implementation notes:
- "Workers" are renamed to "executors" now
- The new "worker" instance holds an "executor" instance and its
related stuffs such as the snapshotter
- The default worker is "runc-overlay"
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-12 15:17:58 +09:00
b7664f71b9
Merge pull request #201 from tonistiigi/containerd-cancellation
...
worker: fix containerd cancellation and cleanup
2017-12-11 12:36:19 +09:00
40a0581853
worker: setup own resolvconf and hosts
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-10 18:23:24 -08:00
61573b1868
worker: fix containerd cancellation and cleanup
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-10 18:22:59 -08:00
fdfa6bcb59
vendor: update containerd to 1.0.0
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-04 23:34:25 -08:00
7699b1a1eb
vendor: update containerd to 9649a428e
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-17 19:43:55 -08:00
6c076f5d6a
cleanup linter warnings
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-16 18:09:35 -08:00
c71a1ca768
bump up containerd (v1.0.0-beta.3)
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-11-13 05:36:59 +00:00
7d0fea5b60
Merge pull request #134 from tonistiigi/gateway-frontend
...
load frontends from images
2017-10-04 14:38:51 +09:00
80bc5f6097
fix exporter attributes interface returns
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-03 21:14:13 -07:00
2809d01cf6
frontend: add gateway frontend / image frontend support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-03 21:14:03 -07:00
f706b13a3f
remove generateID duplicates
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-03 16:13:35 -07:00
Tonis Tiigi
Akihiro Suda
Tibor Vass
Wei Fu
Akihiro Suda
Edgar Lee
Akihiro Suda
John Howard
Tibor Vass
Anda Xu
Michael Crosby
Kunal Kushwaha
Derek McGowan
Ian Campbell
Jess Frazelle