29e5bef684
solver: split llbbridge to a separate file (TODO: separate pkg)
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-13 14:47:09 +09:00
7a5390f355
worker: add support for custom user
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-12 10:26:30 -08:00
c3aa849014
multi-worker daemon
...
- [X] put multiples workers in a single binary ("-tags containerd standalone")
- [X] add worker selector to LLB vertex metadata
- [X] s/worker/executor/g
- [X] introduce the new "worker" concept https://github.com/moby/buildkit/pull/176#discussion_r153693928
- [X] fix up CLI
- [X] fix up tests
- allow using multiples workers (requires inter-vertex cache copier, HUGE!) --> will be separate PR
Implementation notes:
- "Workers" are renamed to "executors" now
- The new "worker" instance holds an "executor" instance and its
related stuffs such as the snapshotter
- The default worker is "runc-overlay"
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-12-12 15:17:58 +09:00
438c850a16
dockerfile: add original command metadata to llb
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-07 16:13:50 -08:00
68165c07f4
exporter: allow exporting config with no snapshots
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-04 23:18:09 -08:00
e0af9d1218
source: add more properties to http source
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-04 12:00:46 -08:00
59628fb20f
add http source
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-02 19:52:15 -08:00
6c076f5d6a
cleanup linter warnings
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-16 18:09:35 -08:00
b888b38e5a
solver: implement IgnoreCache LLB Metadata
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-11-07 03:40:07 +00:00
dfb786a292
Merge pull request #143 from AkihiroSuda/refactor-ref
...
cache, solver: refactor Reference interface
2017-11-06 14:39:15 -08:00
565d08914f
cache, solver: refactor Reference interface
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-11-06 06:30:16 +00:00
65dc07eb28
report imported cache lookup progress
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-15 20:41:47 +02:00
914ea2110b
cache: include os/arch in cache checksum
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-15 19:04:53 +02:00
7171f5a27d
cache: fix root snapshot cache lookup
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-15 19:00:23 +02:00
014a2dd8a1
cache: cache importing from a ref
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-13 15:38:08 -07:00
56e1f71e6b
cache: allow pushing exported cache
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-13 15:37:55 -07:00
1b3f191b37
cache: base code for cache importing
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-12 17:53:28 -07:00
8e611643f7
cache: export cache to content store
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-12 15:04:33 -07:00
0fd670fb73
solver: allow access to cached references for exporting
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-12 11:23:42 -07:00
0ee59e2bdf
Merge pull request #137 from tonistiigi/contentmask
...
solver: replace contentkey with contentmask
2017-10-07 09:47:38 +09:00
2716524dee
exclude selector from content based input hash
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-06 15:43:48 -07:00
53018a8113
solver: replace contentkey with contentmask
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-05 19:58:05 -07:00
59910481ca
solver: make nested build use solverequest
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-05 17:51:04 -07:00
23a114a977
solver: simplify solver public api
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-05 10:05:38 -07:00
80bc5f6097
fix exporter attributes interface returns
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-03 21:14:13 -07:00
2809d01cf6
frontend: add gateway frontend / image frontend support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-03 21:14:03 -07:00
f89ffc2ab4
llb: per-vertex metadata (e.g. IgnoreCache)
...
cache invalidation itself it not implemented yet
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-10-03 05:36:24 +00:00
4a9142223d
solver: fix possible data races
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-27 22:38:54 -07:00
f41283caed
solver: fix shared request progress and cancellation
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-27 12:45:47 -07:00
05c39612d8
dockerfile: resolve to digest ref in frontend
...
Less hacky than keeping the cached resolver.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-22 15:49:37 -07:00
d258bcab7b
dockerfile: clean up converter
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-22 10:49:19 -07:00
cc1b4140af
bgfunc: safer locking
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-19 12:29:29 -07:00
01aaf130e5
solver: refactor to a shared vertex solver model
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-18 13:58:33 -07:00
fd0f7ee2a8
frontend: fix dockerfile frontend resolver
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-15 18:09:28 -07:00
4c22f6c25a
frontend: share attributes to exporter
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-15 18:09:28 -07:00
a26eba7cda
fix for read-only mount
...
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-09-14 06:42:12 +00:00
6d335c3c86
imagesource: share config resolver with frontend
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-05 13:11:27 -07:00
4040029810
dockerfile: improve copy implementation
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-01 19:27:54 -07:00
2d290db465
frontend: base of dockerfile frontend
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-31 23:27:25 -07:00
eb7b5bb76a
contenthash: use selector for content path
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-21 17:07:24 -07:00
8162e15787
llb: add support for mounting subpath or reference
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-07 18:45:50 -07:00
9f6d9a9e78
exporter: add local exporter
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-04 17:12:19 -07:00
8738929b8c
solver: implement content based cache support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-08-04 10:03:26 -07:00
39b9dec52f
Merge pull request #87 from tonistiigi/snapshot-metadata
...
cache: add more metadata to snapshots
2017-07-27 09:36:10 +09:00
5ecab5e142
Merge pull request #11 from dmcgowan/containerd-worker
...
Containerd worker
2017-07-26 14:36:51 -07:00
38ac00090c
cache: add more metadata values to snapshots
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-26 11:28:20 -07:00
9ce5053a8b
Update containerd version
...
Fix for logrus rename, use fork until fixed in moby.
Removed unused tar stream.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-26 11:07:45 -07:00
93757aef7f
nested llb builds support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-24 18:09:16 -07:00
2e89d2dc72
llb: add custom types for input indexes
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-21 16:39:25 -07:00
8b09e89dbb
solver: avoid removing references required for future cache
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-20 17:05:37 -07:00
97b8395272
solver: always cache the original reference instance
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-19 11:08:49 -07:00
ff951eecd9
llb: add readonly mounts support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-18 16:07:37 -07:00
eac79f7c7e
cache: refactor reference reuse and caching
...
Replaces previous mutable.Freeze logic with
commits that can live together with mutable data.
Finalize method is added if the implementation
needs to make sure that the immutable ref is
flushed to the driver. Refs are automaitcally
finalized when writable layers are created on
top of them.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-18 16:01:01 -07:00
5c2e675d18
source: add local file source fundamentals
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-12 23:23:20 -07:00
29d912fddf
exporter: fix exporter progress reporting
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 23:29:21 -07:00
45bfdd5e42
exporter: add image exporter
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 23:29:14 -07:00
2be8499bdc
llb: add scratch support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 22:19:05 -07:00
fa7ee2e2a8
source: add keepgitdir option for git source
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 22:19:05 -07:00
b65989a5e4
source: add git source
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 22:19:05 -07:00
bcb950915b
llb: add const for root mount
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 21:54:38 -07:00
2bb4c5d487
worker: change mounts map to slice
...
Fixes random mount order.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-10 19:18:19 -07:00
25ba9d72de
solver: first pass of instruction cache
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-06 20:42:17 -07:00
c9e2493f06
source: update interface for cache calls
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-06 12:12:17 -07:00
f0c14879c7
solver: remove llb dependency
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-05 22:42:33 -07:00
6f7452ddf1
Merge pull request #49 from tonistiigi/du-update
...
Fixes for du view and releasing
2017-07-03 16:09:11 +09:00
daefcdc9f7
solver: move ops to separate files
...
Preparation for making these unit testable.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-02 21:26:57 -07:00
246ce5e037
cache: fix reference releasing
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-01 08:37:30 -07:00
f9bd9b5cb3
solver: fix vertex info becoming lost on progress replay
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-29 22:42:00 -07:00
65c2f8ef4c
solver: cache vertex progress for replays
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-29 22:42:00 -07:00
2df68a5706
solver: add error reporting and canceling support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-29 15:31:08 -07:00
ec5c60283e
solver: add vertex sharing on concurrent builds
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-28 21:52:05 -07:00
b573060eea
project: remove poc label
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-23 09:37:36 -07:00
728de510f3
llb: improve llb generation
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-21 15:21:36 -07:00
3d3190573c
client: save logs to trace file
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-19 14:32:51 -07:00
0a7a2c1da3
solver: update exec ops to separate functions
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-19 14:32:50 -07:00
4ed7fa3280
progress: refactor progress IDs to items
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-19 14:32:50 -07:00
13f3a45b1d
progress: refactor multireader and add status display
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-16 16:10:42 -07:00
4b41158252
solver: control vertexstatus for pull
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-15 16:08:45 -07:00
177aac4018
api: fix types in protobuf
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-13 23:35:35 -07:00
12a84368df
solver: vertex progress reporting
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-13 23:35:29 -07:00
62b7d04d01
control: base of status reporting
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-13 22:52:59 -07:00
94d62fd432
Add context to functions which call snapshotter
...
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-06-13 16:10:31 -07:00
84a14f44ed
llb: ops execution example
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-09 14:49:01 -07:00
bf8f1ee37a
control: add simple solve method
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-09 14:49:01 -07:00
d90d3cc683
solver: move client to separate pkg
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-09 14:49:01 -07:00
8719505435
solver: add release
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-09 14:49:00 -07:00
25bc7e02fc
solver: base implementation
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-09 14:48:44 -07:00
Akihiro Suda
Tonis Tiigi
Akihiro Suda
Derek McGowan