c3aa849014
multi-worker daemon
...
- [X] put multiples workers in a single binary ("-tags containerd standalone")
- [X] add worker selector to LLB vertex metadata
- [X] s/worker/executor/g
- [X] introduce the new "worker" concept https://github.com/moby/buildkit/pull/176#discussion_r153693928
- [X] fix up CLI
- [X] fix up tests
- allow using multiples workers (requires inter-vertex cache copier, HUGE!) --> will be separate PR
Implementation notes:
- "Workers" are renamed to "executors" now
- The new "worker" instance holds an "executor" instance and its
related stuffs such as the snapshotter
- The default worker is "runc-overlay"
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-12 15:17:58 +09:00
40a0581853
worker: setup own resolvconf and hosts
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-10 18:23:24 -08:00
e4f6a5e752
set more tests to parallel
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-08 18:19:08 -08:00
fdfa6bcb59
vendor: update containerd to 1.0.0
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-04 23:34:25 -08:00
59628fb20f
add http source
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-02 19:52:15 -08:00
e1ced023df
update intergration tests for reuse
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-29 17:42:28 -08:00
541d68955e
override containerd client to set content roots
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-14 13:28:01 -08:00
80f8ee07ab
exporter: export compressed blobs
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-13 16:15:45 -08:00
c71a1ca768
bump up containerd (v1.0.0-beta.3)
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-11-13 05:36:59 +00:00
51d1c4166f
session: expose image auth
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-20 12:56:06 +02:00
56e1f71e6b
cache: allow pushing exported cache
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-13 15:37:55 -07:00
1b3f191b37
cache: base code for cache importing
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-12 17:53:28 -07:00
8e611643f7
cache: export cache to content store
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-12 15:04:33 -07:00
897291de2a
blobs: move blobs differ to common function
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-12 11:24:15 -07:00
23a114a977
solver: simplify solver public api
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-05 10:05:38 -07:00
2809d01cf6
frontend: add gateway frontend / image frontend support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-03 21:14:03 -07:00
f41283caed
solver: fix shared request progress and cancellation
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-27 12:45:47 -07:00
279940b60d
vendor containerd d1e11f17ec7b325f89608dd46c128300b8727d50
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-09-14 19:06:31 +00:00
a26eba7cda
fix for read-only mount
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-09-14 06:42:12 +00:00
a0c9d7c690
vendor containerd v1.0.0-beta.0
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-09-09 11:13:02 +00:00
6d335c3c86
imagesource: share config resolver with frontend
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-05 13:11:27 -07:00
2d290db465
frontend: base of dockerfile frontend
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-31 23:27:25 -07:00
8d57fcffde
vendor containerd 8095244c26fa2daaef850be862e5b1b56d7cec66
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-08-25 10:17:49 -07:00
9f6d9a9e78
exporter: add local exporter
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-04 17:12:19 -07:00
39b9dec52f
Merge pull request #87 from tonistiigi/snapshot-metadata
...
cache: add more metadata to snapshots
2017-07-27 09:36:10 +09:00
5ecab5e142
Merge pull request #11 from dmcgowan/containerd-worker
...
Containerd worker
2017-07-26 14:36:51 -07:00
e3c91ee898
buildctl: add filter to buildctl df
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-26 11:28:26 -07:00
38ac00090c
cache: add more metadata values to snapshots
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-26 11:28:20 -07:00
e460a3926c
Add implementation for containerd worker
...
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-26 11:07:45 -07:00
9ce5053a8b
Update containerd version
...
Fix for logrus rename, use fork until fixed in moby.
Removed unused tar stream.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-26 11:07:45 -07:00
93757aef7f
nested llb builds support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-24 18:09:16 -07:00
ff951eecd9
llb: add readonly mounts support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-18 16:07:37 -07:00
eac79f7c7e
cache: refactor reference reuse and caching
...
Replaces previous mutable.Freeze logic with
commits that can live together with mutable data.
Finalize method is added if the implementation
needs to make sure that the immutable ref is
flushed to the driver. Refs are automaitcally
finalized when writable layers are created on
top of them.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-18 16:01:01 -07:00
6896b5c414
Merge pull request #71 from tonistiigi/local-session
...
source: add local file source fundamentals
2017-07-13 15:56:19 +09:00
5c2e675d18
source: add local file source fundamentals
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-12 23:23:20 -07:00
9e9df712ec
Updates for containerd alpha0
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-12 23:15:12 -07:00
b4f8ac0cba
remove unused code
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-11 00:11:45 -07:00
45bfdd5e42
exporter: add image exporter
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 23:29:14 -07:00
b65989a5e4
source: add git source
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 22:19:05 -07:00
2bb4c5d487
worker: change mounts map to slice
...
Fixes random mount order.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 19:18:19 -07:00
25ba9d72de
solver: first pass of instruction cache
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-06 20:42:17 -07:00
f0c14879c7
solver: remove llb dependency
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-05 22:42:33 -07:00
44415841c9
make blobmapping use metadata package
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-04 21:55:27 -07:00
2df68a5706
solver: add error reporting and canceling support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-29 15:31:08 -07:00
b573060eea
project: remove poc label
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-23 09:37:36 -07:00
e940830845
hack: add binary targets
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-19 16:10:09 -07:00
5244a14bc2
client: add integration tests
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-19 14:35:57 -07:00
3d3190573c
client: save logs to trace file
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-19 14:32:51 -07:00
4b41158252
solver: control vertexstatus for pull
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-15 16:08:45 -07:00
dd9490fb85
control: unbreak standalone
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-13 23:35:36 -07:00
177aac4018
api: fix types in protobuf
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-13 23:35:35 -07:00
12a84368df
solver: vertex progress reporting
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-13 23:35:29 -07:00
62b7d04d01
control: base of status reporting
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-13 22:52:59 -07:00
94d62fd432
Add context to functions which call snapshotter
...
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-06-13 16:10:31 -07:00
f330f4b42b
Use containerd client and default namespace
...
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-06-13 14:51:06 -07:00
84a14f44ed
llb: ops execution example
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-09 14:49:01 -07:00
bf8f1ee37a
control: add simple solve method
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-09 14:49:01 -07:00
10c0daba25
control: add containerd based version
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-09 14:49:01 -07:00
804510e342
control: add default control server
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-09 14:49:01 -07:00
45bcca5c80
control: add diskusage service
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-09 14:49:01 -07:00
28045f19bf
worker: add stdout/stderr to runc worker
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-02 10:20:20 -07:00
08134677cd
worker: runc worker exection
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-02 10:20:05 -07:00
f4695e7b4e
blobmapping: account blob size in disk usage
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-05-31 16:52:53 -07:00
1c5dbe562a
cachemanager: size reporting in diskusage
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-05-31 16:45:04 -07:00
490ac15654
snapshot: add tracking for content blobs
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-05-31 15:09:48 -07:00
fdde46f7ff
cachemanager: check pulled snapshot
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-05-30 14:17:04 -07:00
ae05d02379
Renames and refactoring
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-05-26 23:12:13 -07:00
35e661faf3
sources: basic pull code
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-05-26 22:10:26 -07:00
58c113167f
control: dependencies for pulling with containerd
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-05-26 15:15:20 -07:00
Akihiro Suda
Tonis Tiigi
Akihiro Suda
Derek McGowan