JohnHammond
/
buildkit
mirror of https://github.com/JohnHammond/buildkit.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,565 Commits
7 Branches
72 Tags
26 MiB
Commit Graph

158 Commits (6e1d460d4cf756298c63e3766afd5c3a51b4d636)

Author SHA1 Message Date
Tonis Tiigi 81a5fa5a2e llbsolver: fileop implementation 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2019-03-15 16:22:35 -07:00
Tonis Tiigi 5b4841f308 llb: initial fileop implementation 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2019-03-15 16:22:01 -07:00
Tonis Tiigi f472cb193f exec: fix platform value in cache key 
Use vertex platform instead of the current system one. This shouldn't break any cache unless qemu was used. Even when qemu was used the issue would not cause any cache collisions because the base images for run were scoped by platform, but better to stop using invalid data for cache key calculation.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2019-02-28 11:17:10 -08:00
Akihiro Suda 26010a4864 decouple SessionManager from Worker 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2019-02-23 22:55:17 +09:00
Tonis Tiigi 7d41912f36 remotecache: keep layer description 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2019-01-21 19:39:15 -08:00
Tonis Tiigi ac4037a1d9 solver: keep original cache creation date when copying between stores 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2019-01-18 11:19:34 -08:00
Akihiro Suda c54f4a986d support --oci-worker-no-process-sandbox 
Note that this mode allows build executor containers to kill (and potentially ptrace) an arbitrary process in the BuildKit host namespace.
This mode should be enabled only when the BuildKit is running in a container as an unprivileged user.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2019-01-08 10:42:52 +09:00
Akihiro Suda a62e7cf1e0 worker/base: allow running without `git` installed 
genuinetools/img#195

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-11-28 18:43:43 +09:00
Tonis Tiigi ddfb7f77c9 hack: update test dockerfiles to buildkit 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-10-10 23:26:05 -07:00
Tonis Tiigi 89e8097d4c fix setting uncompressed label on content 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-10-08 23:11:25 -07:00
Edgar Lee 688972691f Fix typo in error getting diff pairs 2018-09-21 13:56:05 -07:00
Akihiro Suda aa73951164
Merge pull request #631 from jhowardmsft/boltdb 
Revendoring to move boltdb to bbolt
 2018-09-19 13:59:08 +09:00
John Howard 2de2c04c8e Revendoring to move boltdb to bbolt 
Signed-off-by: John Howard <jhoward@microsoft.com>
 2018-09-18 11:18:08 -07:00
Tonis Tiigi 757795e556 cache: fix usage counters 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-09-14 18:44:44 -07:00
Akihiro Suda 907411c7e8
Merge pull request #605 from fuweid/feature_add_namespace_in_config 
cmd,worker: support customize namespace for containerd worker
 2018-09-11 11:56:24 +09:00
Wei Fu 53aa4053bb cmd,worker: support customize namespace for containerd worker 
Allow user to customize containerd namespace, not just `buildkit`. It
can help user to integration with existing data easily.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2018-09-11 09:41:42 +08:00
Tonis Tiigi ecd4a22546 resolver: add config support for mirrors/plainhttp 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-09-07 18:44:45 -07:00
Tibor Vass a9fe50acf1
Merge pull request #580 from AntaresS/handle-cgroups 
allow customized cgroup-parent for runcexecutor
 2018-09-06 11:15:51 -07:00
Anda Xu b0677e7ef1 allow customized cgroup-parent for runcexecutor 
Signed-off-by: Anda Xu <anda.xu@docker.com>
 2018-09-05 12:53:57 -07:00
Tonis Tiigi 079c747c4e ontrol: support calling automatic GC 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-09-04 18:13:29 -07:00
Michael Crosby b97bc71adb Refactor networking with ns paths 
This fixes the issues where buildkit and callers do not have to be a
subpreaper in order to use networking.  I can add CNI provider later,
with a hidden sub command to create a new network namespace and bind
mount it to buildkit's state dir.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
 2018-08-21 13:37:47 -04:00
Tonis Tiigi f8dd602282 runc: improve canceling 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-08-07 11:51:56 -07:00
Kunal Kushwaha 765f1b64b9 executor: allow network providers 
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-08-07 11:51:56 -07:00
Tonis Tiigi d1ad6961e3 cache: detect refs shared with image store 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-27 14:08:50 -07:00
Tibor Vass aa81807f1c
Merge pull request #542 from tonistiigi/resolvemode 
llb: add resolvemode to image source
 2018-07-26 14:04:27 -07:00
Tonis Tiigi 87f43efda5 cache: implementation for prune filtering 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-25 19:35:49 -07:00
Tonis Tiigi 9f5e5c7dc7 metaresolver: refactor to option struct and resolvemode 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-25 18:22:15 -07:00
Tonis Tiigi a7fedd9c9f control: clean up cache metadata after prune 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-23 16:27:01 -07:00
Tonis Tiigi 2da2aff164 llbsolver: add secret mounts 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-19 22:02:05 -07:00
Tonis Tiigi bc765861be diff: implement windows layer support for linux 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-16 16:33:21 -07:00
Tonis Tiigi 946a12eb42 gateway: expose worker information 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-09 16:32:39 -07:00
Tõnis Tiigi f3b743bd13
Merge pull request #467 from dmcgowan/export-oci-media-types 
Support creating and pushing OCI types
 2018-06-28 14:46:34 -07:00
Derek McGowan 618f34a1f9 Support creating and pushing OCI types 
Adds image and oci exporter option "oci-mediatypes"
Ensures that the images created in the content store
have the correct type which matches the manifest.
Sets the correct media type on the descriptor in push from
reading the type specified in the manifest.
Removes use of distribution manifest packages.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
 2018-06-28 11:23:35 -07:00
Tonis Tiigi f9383c48a3 source: image source platform support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-06-25 10:39:17 -07:00
Akihiro Suda 19612b901f
Merge pull request #462 from tonistiigi/llbconstraints 
llb: force platform in llb and allow constraints
 2018-06-26 01:03:38 +09:00
Tonis Tiigi 85e98104cf worker: remove arch/os labels 
Superceded by platforms array.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-06-24 22:49:06 -07:00
Tonis Tiigi 242697af59 llb: add marshal roundtrip tests for platform 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-06-22 17:55:36 -07:00
Tonis Tiigi a1bafadfd5 worker: add platforms support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-06-22 11:46:46 -07:00
Kunal Kushwaha e991a846be Default mounts set for buildkit 
containerd/oci_specs mounts /run, which is not
required by buildkit.

Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
 2018-06-07 14:43:44 +09:00
Akihiro Suda adef0dedef oci-worker: experimental support for rootless mode 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-05-31 16:05:13 +09:00
Tonis Tiigi 8084e8cde9 llbsolver: allow empty returns for opts 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-22 15:46:32 -07:00
Tonis Tiigi 2bd78c3c6e llbsolver: add support for cached mount type 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-15 12:24:20 -07:00
Tonis Tiigi 71f5e6dd21 solver: rename solver-next to solver 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi f00ba89a30 llbsolver: support detecting existing remotes 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi a7bc9b9fd2 cache: maintain creation time with remote cache 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi 77c2793ebb remove unused packages 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi b6c782d53e control: switch to use solver-next 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi e24e194078 remove net/context references 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-04-30 18:10:54 -07:00
Tonis Tiigi 2dba0488c8 runc: make command name configurable 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-04-17 20:39:19 -07:00
Tõnis Tiigi 89f8bc1c02
Merge pull request #329 from AkihiroSuda/containerd-20180403 
update containerd (binary: v1.0.3, library: Apr 3, 2018)
 2018-04-04 10:05:42 -07:00
Akihiro Suda 126f91dea7 update containerd (binary: v1.0.3, library: Apr 3, 2018) 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-04-04 17:22:20 +09:00
Ian Campbell 4bae684e32 Invert tags on runc worker test so they run by default. 
Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-04-03 11:01:46 +01:00
Tõnis Tiigi a0a7301ea0
Merge pull request #312 from tonistiigi/throttle-gc 
worker: throttle calls to gc
 2018-03-26 09:34:35 -07:00
Jess Frazelle b0f34cb8cd
export cache 
Signed-off-by: Jess Frazelle <acidburn@microsoft.com>
 2018-03-21 12:34:05 -04:00
Tonis Tiigi da633a8e6b worker: throttle calls to gc 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-03-19 10:40:31 -07:00
Tõnis Tiigi 18280e73cc
Merge pull request #289 from ijc/allow-local-image-for-gateway-fe 
Use local images if pull fails
 2018-03-06 09:59:17 -08:00
Ian Campbell ac5091c10b Use local images if pull fails 
When using `--frontend=gateway.v0` in order to use a tagged image as the
frontend it is currently required to push the tag to a registry, so that
buildkit can pull it. One can use the `image@sha256:digest` form but this is
inconvenient for the development cycle.

Instead, introduce a proxy `Resolver` which if the pull fails checks the local
image store (if there is one, which is worker dependent) for a suitable image
and if so uses that.

With this I can now:

    buildctl build --frontend=dockerfile.v0 --local context=. --local dockerfile=. --exporter=image --exporter-opt=name=docker.io/ijc25/fe:dev
    builtctl build --frontend=gateway.v0 --frontend-opt=source=ijc25/fe:dev --exporter=image --exporter-opt=name=hello-world:dev

Without any push to a registry.

I am using the containerd worker.

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-03-06 12:19:34 +00:00
Akihiro Suda de73349f01 worker/oci: allow specifying arbitrary snapshotter factory 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-02-26 18:15:10 +09:00
Akihiro Suda 622d4a442a worker/oci: allow using naive snapshotter 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-02-20 12:13:18 +09:00
Tonis Tiigi 7906fb3085 vendor: update containerd to v1.0.2 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-02-13 16:00:09 -08:00
Tonis Tiigi 6e40e83d35 Remove net/context dependencies 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-01-17 09:11:22 -08:00
Tonis Tiigi b0679c66db snapshot: clean up snapshot interface 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-01-04 23:09:05 -08:00
Tonis Tiigi 56353f99a8 snapshot: combine nogc snapshotting helpers 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-01-04 23:09:05 -08:00
Tonis Tiigi 63ce643468 cache: add prune support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-01-04 23:09:05 -08:00
Akihiro Suda 84c0dd0b07 worker: add labels 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-20 16:28:47 +09:00
Akihiro Suda dca082582e worker, solver: s/ResolveVertex/ResolveOp/g 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-19 13:20:30 +09:00
Tonis Tiigi 85c0f99ba0 exporter: add Docker compatible exporter 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-18 12:19:17 -08:00
Akihiro Suda 21bebaea80 solver, worker: minor cleanup 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-18 16:20:59 +09:00
Tonis Tiigi efde4f2340 worker, solver: update interfaces 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-18 16:20:59 +09:00
Akihiro Suda 2a33b2e822 *: update build tags (`containerd` -> `no_oci_worker`) 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-18 15:47:56 +09:00
Akihiro Suda a24ab34522 buildctl: fix dump-metadata 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-15 16:44:09 +09:00
Akihiro Suda 682ac0248c
Merge pull request #211 from tonistiigi/ociexport 
exporter: add OCI exporter
 2017-12-14 11:05:08 +09:00
Tonis Tiigi 6edccb7913 exporter: add OCI exporter 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-13 09:57:28 -08:00
Tonis Tiigi 8840dc7439 image: separate imagewriter to own struct 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-12 22:12:35 -08:00
Akihiro Suda 89450211fa *: deduplicate instructioncache definitions 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-13 15:03:14 +09:00
Tonis Tiigi e5e24af1f5 runc: change runc test to require 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-12 14:59:24 -08:00
Akihiro Suda c3aa849014 multi-worker daemon 
- [X] put multiples workers in a single binary ("-tags containerd standalone")
- [X] add worker selector to LLB vertex metadata
- [X] s/worker/executor/g
- [X] introduce the new "worker" concept https://github.com/moby/buildkit/pull/176#discussion_r153693928
- [X] fix up CLI
- [X] fix up tests
- allow using multiples workers (requires inter-vertex cache copier, HUGE!) --> will be separate PR

Implementation notes:
- "Workers" are renamed to "executors" now
- The new "worker" instance holds an "executor" instance and its
related stuffs such as the snapshotter
- The default worker is "runc-overlay"

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-12 15:17:58 +09:00
Akihiro Suda b7664f71b9
Merge pull request #201 from tonistiigi/containerd-cancellation 
worker: fix containerd cancellation and cleanup
 2017-12-11 12:36:19 +09:00
Tonis Tiigi 40a0581853 worker: setup own resolvconf and hosts 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-10 18:23:24 -08:00
Tonis Tiigi 61573b1868 worker: fix containerd cancellation and cleanup 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-10 18:22:59 -08:00
Tonis Tiigi fdfa6bcb59 vendor: update containerd to 1.0.0 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-04 23:34:25 -08:00
Tonis Tiigi 7699b1a1eb vendor: update containerd to 9649a428e 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-11-17 19:43:55 -08:00
Tonis Tiigi 6c076f5d6a cleanup linter warnings 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-11-16 18:09:35 -08:00
Akihiro Suda c71a1ca768 bump up containerd (v1.0.0-beta.3) 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-11-13 05:36:59 +00:00
Akihiro Suda 7d0fea5b60 Merge pull request #134 from tonistiigi/gateway-frontend 
load frontends from images
 2017-10-04 14:38:51 +09:00
Tonis Tiigi 80bc5f6097 fix exporter attributes interface returns 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-03 21:14:13 -07:00
Tonis Tiigi 2809d01cf6 frontend: add gateway frontend / image frontend support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-03 21:14:03 -07:00
Tonis Tiigi f706b13a3f remove generateID duplicates 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-03 16:13:35 -07:00
Tonis Tiigi 01aaf130e5 solver: refactor to a shared vertex solver model 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-18 13:58:33 -07:00
Akihiro Suda a0c9d7c690 vendor containerd v1.0.0-beta.0 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-09-09 11:13:02 +00:00
Akihiro Suda 8d57fcffde vendor containerd 8095244c26fa2daaef850be862e5b1b56d7cec66 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-08-25 10:17:49 -07:00
Tonis Tiigi 8162e15787 llb: add support for mounting subpath or reference 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-08-07 18:45:50 -07:00
Akihiro Suda 03f9eee2d0 Fix windows build 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-08-04 06:24:31 +00:00
Derek McGowan e460a3926c Add implementation for containerd worker 
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
 2017-07-26 11:07:45 -07:00
Derek McGowan 9ce5053a8b Update containerd version 
Fix for logrus rename, use fork until fixed in moby.
Removed unused tar stream.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
 2017-07-26 11:07:45 -07:00
Tonis Tiigi ff951eecd9 llb: add readonly mounts support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-18 16:07:37 -07:00
Tonis Tiigi b65989a5e4 source: add git source 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-10 22:19:05 -07:00
Tonis Tiigi 2bb4c5d487 worker: change mounts map to slice 
Fixes random mount order.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-10 19:18:19 -07:00
Tonis Tiigi dd4c575236 buildd: graceful shutdown on termination signals 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-03 00:13:06 -07:00
Tonis Tiigi 2df68a5706 solver: add error reporting and canceling support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-29 15:31:08 -07:00