JohnHammond
/
buildkit
mirror of https://github.com/JohnHammond/buildkit.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,515 Commits
7 Branches
72 Tags
27 MiB
Commit Graph

341 Commits (59edca02f388b6bd13a93a7df868129738710092)

Author SHA1 Message Date
Tõnis Tiigi 1508ae0fc7
Merge pull request #608 from tonistiigi/ssh-forwarding 
sshforward: implement ssh socket forwarding
 2018-09-11 08:01:58 -07:00
Tonis Tiigi a2db26227f solver: fix data race 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-09-07 17:17:17 -07:00
Tonis Tiigi 6888956557 sshforward: implement ssh socket forwarding 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-09-07 08:57:06 -07:00
Tõnis Tiigi e3bc522f7c
Merge pull request #582 from ijc/gateway-client-snapshot-leak 
clientside-frontend: release non-result refs after completion
 2018-08-31 09:22:28 -07:00
Tibor Vass 9d1b0e6b31
Merge pull request #597 from tonistiigi/graceful-scheduler-errors 
solver: gracefully handle scheduler bugs
 2018-08-30 10:56:39 -07:00
Tibor Vass 7e971435a4
Merge pull request #590 from tonistiigi/fix-panic 
solver: fix panic on creating input requests
 2018-08-30 10:54:41 -07:00
Tõnis Tiigi 45aedff363
Merge pull request #571 from ijc/solver-def-and-frontend-mutually-exclusive 
solver: Don't solve both the provided Defininiton & frontend
 2018-08-30 10:10:52 -07:00
Ian Campbell 59391d1647 clientside-frontend: release non-result refs after completion 
Tests such as TestNoSnapshotLeak were failing in client mode (e.g. using #522)
because we weren't releasing the intermediate refs.

Resolve this by refactoring the existing code which frees the intermediate refs
from `gatewayFrontend.Solve` into a method on `llbBridgeForwarder` and as well
as the original site also call from the solver when the top-level solve (in
clientside frontend mode) completes. The original call (which is via a defer)
could likely sensibly be moved either earlier or later if desired but leave it
here it is to minimise the scope of the change.

The previous code used the `retErr` named return but the code between that
point and the end of the function already ensured that `lbf.err` is the same as
`retErr`, thus the only change in the code which has moved is
`s/retErr/lbf.err/`. Note that the `res` named return was previously unused by
name.

Fixes #581.

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-08-30 14:54:50 +01:00
Ian Campbell cc83b34572 solver: remove redundant conditional 
There is now no way to reach this point with the test being false.

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-08-30 14:01:27 +01:00
Ian Campbell ab7928edff solver: error if both Definition and Frontend are given. 
Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-08-30 13:59:26 +01:00
Tonis Tiigi 0d68543b1e solver: mark build failed instead of panicking on scheduler error 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-08-29 11:10:05 -07:00
Tonis Tiigi c77b6d4759 solver: gracefully handle bugs leaving incoming requests open 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-08-28 17:07:50 -07:00
Tonis Tiigi a384bccbbf solver: add test for input request deadlock 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-08-28 16:47:31 -07:00
Tonis Tiigi ee82ffe538 solver: correct dep key conditions 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-08-27 16:18:23 -07:00
Tonis Tiigi 1127380f91 solver: allow enabling debugger on runtime 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-08-27 16:16:35 -07:00
Tonis Tiigi 23904e6ffa solver: add test for input request deadlock 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-08-27 16:12:50 -07:00
Ian Campbell fdf13cf96e Allow some Metadata to be return from frontend to the control client 
Propagate anything in the `frontend.*` namespace from the `frontend.Result`
into the `client.SolveResponse`.

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-08-17 12:45:21 +01:00
Akihiro Suda af46188e9b
Merge pull request #533 from ijc/client-gateway 
access gateway API from client
 2018-08-17 05:12:23 +09:00
Ian Campbell e05be2fd2c solver: Don't solve both the provided Defininiton & frontend 
This seems wasteful. I think there is no side-effects of the Load+Build of the
definition which would make a difference to the frontend solve.

Note that this changes the presedence of the actual result, previously the
frontend result would overwrite the definition's one, while now the frontend
would not be run.

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-08-15 16:20:02 +01:00
Ian Campbell 22f632f895 allow frontends to be run directly on the control client side 
This allows builder code to be written which can be built as either a gateway
container or in a purely client side configuration, giving implementors more
flexibility.

Now when `Solve` sees a request with neither a definition nor a frontend
specified it will make the job available via new LLBBridge endpoints on the
control socket which the client can then use. These end points require the job
id to be present in the gRPC metadata and a client side object is added to
facilitate this.

The `llbBridgeForwarder` type is now exposed as a public `interface
LLBBridgeForwarder` which satisfies the underlying gRPC server interface
(`pb.LLBBridgeServer`) as well as a new `Done()` & `Result()` pair which can be
used to wait for the client to call `Return()` (using a model similar to
`context.Context`).

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-08-14 11:50:08 +01:00
Tonis Tiigi 95e0348f57 dockerfile: enable global net-mode settings 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-08-09 14:03:40 -07:00
Tonis Tiigi 130f5f5ab0 solver: net host with basic entitlements support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-08-09 14:03:35 -07:00
Tonis Tiigi 4945fe758c llbsolver: add support for extra host records 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-08-03 13:56:13 -07:00
Ian Campbell f7937f1989 solver: avoid dereferencing nil res.Metadata 
I saw this:
    panic: assignment to entry in nil map

    goroutine 3173 [running]:
    github.com/moby/buildkit/exporter/oci.(*imageExporterInstance).Export(0xc42094ac40, 0xecac60, 0xc4211ca690, 0xed12c0, 0xc42000fca8, 0x0, 0x0, 0x0, 0x0, 0x0)
    	/go/src/github.com/moby/buildkit/exporter/oci/export.go:113 +0x156
    github.com/moby/buildkit/solver/llbsolver.(*Solver).Solve.func2(0xecac60, 0xc4211ca690, 0xc4202207c0, 0x0)
    	/go/src/github.com/moby/buildkit/solver/llbsolver/solver.go:132 +0x7d
    github.com/moby/buildkit/solver/llbsolver.inVertexContext(0xecac60, 0xc4211ca690, 0xe2ec38, 0x1d, 0xc420d43400, 0x0, 0x0)
    	/go/src/github.com/moby/buildkit/solver/llbsolver/solver.go:203 +0x1f6
    github.com/moby/buildkit/solver/llbsolver.(*Solver).Solve(0xc4203f7dc0, 0xecac60, 0xc420deb830, 0xc4203ed200, 0x19, 0xc4202ff840, 0x0, 0x0, 0x0, 0x0, ...)
    	/go/src/github.com/moby/buildkit/solver/llbsolver/solver.go:131 +0x761
    github.com/moby/buildkit/control.(*Controller).Solve(0xc42017e870, 0xecac60, 0xc420deb830, 0xc4201edd40, 0xc42017e870, 0x1, 0x1)
    	/go/src/github.com/moby/buildkit/control/control.go:207 +0x4b8
    github.com/moby/buildkit/api/services/control._Control_Solve_Handler.func1(0xecac60, 0xc420deb800, 0xde2180, 0xc4201edd40, 0xecac60, 0xc420deb800, 0xed67a0, 0x1595288)
    	/go/src/github.com/moby/buildkit/api/services/control/control.pb.go:810 +0x86
    github.com/moby/buildkit/vendor/github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1(0xecac60, 0xc420deb800, 0xde2180, 0xc4201edd40, 0xc4202ff9e0, 0xc4202ffa00, 0x0, 0x0, 0xebbea0, 0xc420188310)
    	/go/src/github.com/moby/buildkit/vendor/github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc/server.go:57 +0x2ba
    main.unaryInterceptor.func1(0xecaba0, 0xc4206afe40, 0xde2180, 0xc4201edd40, 0xc4202ff9e0, 0xc4202ffa00, 0x0, 0x0, 0x0, 0x0)
    	/go/src/github.com/moby/buildkit/cmd/buildkitd/main.go:330 +0x15f
    github.com/moby/buildkit/api/services/control._Control_Solve_Handler(0xd76a00, 0xc42017e870, 0xecac60, 0xc420deb530, 0xc42028d030, 0xc420450760, 0x0, 0x0, 0x34, 0x3)
    	/go/src/github.com/moby/buildkit/api/services/control/control.pb.go:812 +0x167
    github.com/moby/buildkit/vendor/google.golang.org/grpc.(*Server).processUnaryRPC(0xc420244700, 0xed39c0, 0xc420694000, 0xc420671680, 0xc420497650, 0x152cef8, 0x0, 0x0, 0x0)
    	/go/src/github.com/moby/buildkit/vendor/google.golang.org/grpc/server.go:1011 +0x4fc
    github.com/moby/buildkit/vendor/google.golang.org/grpc.(*Server).handleStream(0xc420244700, 0xed39c0, 0xc420694000, 0xc420671680, 0x0)
    	/go/src/github.com/moby/buildkit/vendor/google.golang.org/grpc/server.go:1249 +0x1318
    github.com/moby/buildkit/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc42063e0f0, 0xc420244700, 0xed39c0, 0xc420694000, 0xc420671680)
    	/go/src/github.com/moby/buildkit/vendor/google.golang.org/grpc/server.go:680 +0x9f
    created by github.com/moby/buildkit/vendor/google.golang.org/grpc.(*Server).serveStreams.func1
    	/go/src/github.com/moby/buildkit/vendor/google.golang.org/grpc/server.go:678 +0xa1

Which was due to `res, err := s.Bridge(j).Solve(ctx, req)` having `res.Metadata
== nil`. There are several paths in `llbBridge.Solve()` where this can be the
case, plus a case where this comes from a frontend which should not be allowed
to crash the daemon.

Likely introduced by d70d816dee or 6be1257f5d.

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-08-03 16:08:04 +01:00
Tonis Tiigi 4c44d2c6e4 dockerfile: custom names to vertexes 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-31 10:41:30 -07:00
Tonis Tiigi b52c0002db llbsolver: show status of resolving image config 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-27 15:59:11 -07:00
Tonis Tiigi fe3388e70d solver: change builder.call to builder.context 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-27 15:39:14 -07:00
Tonis Tiigi 489246dd28 cache: support for internal/frontend record type 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-26 22:54:53 -07:00
Tonis Tiigi 57b96a0ee5 cache: add record type field to usage record 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-26 22:54:39 -07:00
Tonis Tiigi 9f5e5c7dc7 metaresolver: refactor to option struct and resolvemode 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-25 18:22:15 -07:00
Tonis Tiigi a3b5d22419 llb: add resolvemode to image source 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-25 18:20:04 -07:00
Tonis Tiigi d70d816dee exporter: update export signature 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-25 14:37:01 -07:00
Akihiro Suda 8fa31102d5
Merge pull request #539 from tonistiigi/fix-cache-metadata-release 
Fix cache metadata release
 2018-07-24 13:36:22 +09:00
Akihiro Suda 031c065a06
Merge pull request #536 from tonistiigi/better-errors 
Improve errors on missing sources and hash calculation
 2018-07-24 11:25:58 +09:00
Akihiro Suda 89eee2126e
Merge pull request #535 from tonistiigi/merge-fix 
solver: fix edge merge skipping issue
 2018-07-24 11:25:09 +09:00
Tonis Tiigi 73eaeba88c boltdbcache: fix releasing results 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-23 18:43:32 -07:00
Tonis Tiigi a7fedd9c9f control: clean up cache metadata after prune 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-23 16:27:01 -07:00
Tonis Tiigi 97ea41ee6a solver: improve multi-key root caching 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-23 14:27:35 -07:00
Tonis Tiigi c575077cd6 solver: fix edge merge skipping issue 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-23 10:52:31 -07:00
Tonis Tiigi 354a967470 solver: join slow-cache error with vertex 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-21 13:03:09 -07:00
Tonis Tiigi 08d324d0dd exec: fix secret mounts for rootless 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-19 22:02:11 -07:00
Tonis Tiigi 2da2aff164 llbsolver: add secret mounts 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-19 22:02:05 -07:00
Tonis Tiigi aa3aef3c25 llbsolver: validate llb caps 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-18 17:22:30 -07:00
Tonis Tiigi d4c57aec5d llb: add caps support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-18 17:22:30 -07:00
Tonis Tiigi 303b5da713 llbsolver: use result type with multi ref support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-12 17:52:23 -07:00
Tonis Tiigi 01b7a29e86 gateway: return result from solve 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-12 17:51:44 -07:00
Tonis Tiigi 57006ea333 cache: avoid commit on finalize when possible 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-11 16:21:27 -07:00
Tonis Tiigi ddf3f3e59c frontend: fix two data races 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-10 19:31:33 -07:00
Akihiro Suda dca7bdcd4c
Merge pull request #495 from tonistiigi/unique-local 
llb: set unique ID to local sources without session
 2018-07-10 20:34:04 +09:00
Tonis Tiigi 946a12eb42 gateway: expose worker information 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-09 16:32:39 -07:00
Tonis Tiigi 40e270a2e6 llb: set unique ID to local sources without session 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-07-09 16:27:12 -07:00
Akihiro Suda 80d2f820f9 decopule cache/remotecache from registry 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-07-04 17:38:09 +09:00
Akihiro Suda 1b34bd1e31
Merge pull request #464 from tonistiigi/image-platform 
platform support for image and dockerfile
 2018-06-27 14:36:02 +09:00
Tonis Tiigi e618878451 dockerfile: platform support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-06-25 11:06:08 -07:00
Tonis Tiigi f9383c48a3 source: image source platform support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-06-25 10:39:17 -07:00
Tonis Tiigi b444b9f323 llbsolver: validate runtime platforms for exec op 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-06-25 10:38:03 -07:00
Tonis Tiigi 242697af59 llb: add marshal roundtrip tests for platform 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-06-22 17:55:36 -07:00
Tonis Tiigi 25d975ce6e llb: define platforms types 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-06-22 16:49:03 -07:00
Tonis Tiigi ccbf185006 llbsolver: allow sharing modes for cache mounts 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-06-19 15:57:20 -07:00
Tonis Tiigi 3c973bacfe llbsolver: allow concurrent cache mount access 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-06-08 20:02:30 -07:00
Tonis Tiigi 6f569455d0 dockerfile: enable tmpfs mounts on run —mount 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-06-08 13:42:38 -07:00
Tonis Tiigi 596b03a8a0 llbsolver: allow tmpfs mounts 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-06-08 13:05:14 -07:00
Akihiro Suda 60344aa592
Merge pull request #442 from tonistiigi/runmount 
dockerfile: add run --mount support
 2018-06-09 02:35:34 +09:00
Tonis Tiigi 055dcb6c09 local: allow followpaths for local source 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-06-07 23:23:13 -07:00
Tonis Tiigi 479419abf2 llbsolver: don’t error on readonly scratch mount 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-06-07 16:23:53 -07:00
Tonis Tiigi 1067c24df2 git: support all git transports 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-29 10:56:08 -07:00
Akihiro Suda e2fc1a702d
Merge pull request #414 from tonistiigi/stdio-debug 
exec: make stdio debugging optional
 2018-05-26 06:59:04 +09:00
Tonis Tiigi 6b23ab01ce exec: make stdio debugging optional 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-25 11:15:32 -07:00
Tonis Tiigi 97ddeffdb2 exec: remove debug 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-25 10:45:10 -07:00
Tonis Tiigi eefc49ff72 dockerfile: detect empty images early 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-22 16:54:04 -07:00
Tonis Tiigi 8084e8cde9 llbsolver: allow empty returns for opts 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-22 15:46:32 -07:00
Tonis Tiigi 66488dc6d8 exec: support proxy settings 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-21 16:08:11 -07:00
Tonis Tiigi 2bd78c3c6e llbsolver: add support for cached mount type 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-15 12:24:20 -07:00
Tonis Tiigi 692bdc8aa5 llbsolver: comments for exec op mount handling 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-15 12:09:59 -07:00
Tonis Tiigi fc888e6413 solver: move cachestorage testsuite 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi 71f5e6dd21 solver: rename solver-next to solver 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi 335b1c3660 llbsolver: update for new cache variables 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi 75bc38c15b client: add basic cache import test 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi 77c2793ebb remove unused packages 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-14 09:53:34 -07:00
Tonis Tiigi b80f45955a solver: allow exporters to set data to solve results 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-05-07 20:52:11 -07:00
Akihiro Suda aabfa3b84a
Merge pull request #324 from ijc/readonly-rootfs-execop 
Make llb.ReadonlyRootFS usable with common container images
 2018-04-04 11:43:44 +09:00
Ian Campbell d9b64588cb Regenerate protobuf go file with gogoproto v0.5 
This matches the version in vendor.conf, the previous version was `v0.5~25`
according to `git describe`.

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-04-03 13:45:50 +01:00
Ian Campbell 2e6adc027d Reduce code duplication with a closure. 
Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-04-03 11:13:15 +01:00
Ian Campbell ddae5a6ea4 Make llb.ReadonlyRootFS usable with common container images 
e.g. with busybox image:

    OCI runtime create failed: container_linux.go:348:
    starting container process caused "process_linux.go:402:
    container init caused \"rootfs_linux.go:58:
    mounting \\\"proc\\\" to rootfs \\\"/.../rootfs\\\" at \\\"/proc\\\"
    caused \\\"mkdir /.../rootfs/proc: read-only file system\\\"\"": unknown

This is because we were setting the underlying snapshot readonly so the various
mountpoints (here /proc) cannot be created. This would not be necessary if
those mountpoints were present in images but they typically are not.

The right way to get around this (used e.g. by `ctr`) is to use a writeable
snapshot but to set root readonly in the OCI spec. In this configuration the
rootfs is writeable when mounts are processed but is then made readonly by the
runtime (runc) just before entering the user specified binary within the
container.

This involved a surprising amount of plumbing.

Use this new found ability in the dockerfile converter's `dispatchCopy`
function.

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-04-03 11:04:07 +01:00
Ian Campbell ac6598d255 Check that mounts always have a target. 
Otherwise the daemon panics when generating the OCI spec.

For belt and braces check in the ExecOp Run function but also when generating the spec.

Signed-off-by: Ian Campbell <ijc@docker.com>
 2018-03-22 12:05:12 +00:00
Akihiro Suda ecf2e8dbc9 llb: update docs 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2018-02-27 16:06:01 +09:00
Tonis Tiigi 7906fb3085 vendor: update containerd to v1.0.2 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-02-13 16:00:09 -08:00
Vincent Demeester 79e346f59a
Add deadcode to the linter 
it tends to help keeping uneeded code out O:)

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
 2018-02-01 17:59:04 -08:00
Tonis Tiigi 6e40e83d35 Remove net/context dependencies 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-01-17 09:11:22 -08:00
Tonis Tiigi 29b72a3912 solver: add some trace points 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-01-08 18:03:12 -08:00
Tonis Tiigi b94704e55c session: fix session closing and tracing 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2018-01-08 18:03:06 -08:00
Tonis Tiigi d804819dad progress: fix skipping cached vertex progress 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-20 21:33:36 -08:00
Tonis Tiigi dd49b92fd1 solver: add simple locking to catch content duplicated 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-20 16:42:09 -08:00
Akihiro Suda 84c0dd0b07 worker: add labels 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-20 16:28:47 +09:00
Akihiro Suda dca082582e worker, solver: s/ResolveVertex/ResolveOp/g 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-19 13:20:30 +09:00
Akihiro Suda 21bebaea80 solver, worker: minor cleanup 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-18 16:20:59 +09:00
Tonis Tiigi efde4f2340 worker, solver: update interfaces 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-18 16:20:59 +09:00
Akihiro Suda a4316d16c7 solver: split llbop pkg 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-18 16:20:59 +09:00
Tonis Tiigi e58e6d1175 replace FollowSymlink with fs.RootPath 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-17 20:54:49 -08:00
Tonis Tiigi 2d3f36d359 source: add sharedkeyhint to local source 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-15 15:13:15 -08:00
Tonis Tiigi 08e1c2990c dockerfile: add dockerignore support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-15 15:03:35 -08:00
Tonis Tiigi ad7c6a905c source: add function for parsing llb to identifier 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-13 16:43:49 -08:00
Akihiro Suda ead75c0d95 solver.mergeRemoteCache -> cache/instructioncache.Union 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-13 15:16:46 +09:00
Akihiro Suda 89450211fa *: deduplicate instructioncache definitions 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-13 15:03:14 +09:00
Akihiro Suda 29e5bef684 solver: split llbbridge to a separate file (TODO: separate pkg) 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-13 14:47:09 +09:00
Tonis Tiigi 7a5390f355 worker: add support for custom user 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-12 10:26:30 -08:00
Akihiro Suda c3aa849014 multi-worker daemon 
- [X] put multiples workers in a single binary ("-tags containerd standalone")
- [X] add worker selector to LLB vertex metadata
- [X] s/worker/executor/g
- [X] introduce the new "worker" concept https://github.com/moby/buildkit/pull/176#discussion_r153693928
- [X] fix up CLI
- [X] fix up tests
- allow using multiples workers (requires inter-vertex cache copier, HUGE!) --> will be separate PR

Implementation notes:
- "Workers" are renamed to "executors" now
- The new "worker" instance holds an "executor" instance and its
related stuffs such as the snapshotter
- The default worker is "runc-overlay"

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-12-12 15:17:58 +09:00
Tonis Tiigi 438c850a16 dockerfile: add original command metadata to llb 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-07 16:13:50 -08:00
Tonis Tiigi 68165c07f4 exporter: allow exporting config with no snapshots 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-04 23:18:09 -08:00
Tonis Tiigi e0af9d1218 source: add more properties to http source 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-04 12:00:46 -08:00
Tonis Tiigi 59628fb20f add http source 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-12-02 19:52:15 -08:00
Tonis Tiigi 6c076f5d6a cleanup linter warnings 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-11-16 18:09:35 -08:00
Akihiro Suda b888b38e5a solver: implement IgnoreCache LLB Metadata 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-11-07 03:40:07 +00:00
Tõnis Tiigi dfb786a292
Merge pull request #143 from AkihiroSuda/refactor-ref 
cache, solver: refactor Reference interface
 2017-11-06 14:39:15 -08:00
Akihiro Suda 565d08914f cache, solver: refactor Reference interface 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-11-06 06:30:16 +00:00
Tonis Tiigi 65dc07eb28 report imported cache lookup progress 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-15 20:41:47 +02:00
Tonis Tiigi 914ea2110b cache: include os/arch in cache checksum 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-15 19:04:53 +02:00
Tonis Tiigi 7171f5a27d cache: fix root snapshot cache lookup 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-15 19:00:23 +02:00
Tonis Tiigi 014a2dd8a1 cache: cache importing from a ref 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-13 15:38:08 -07:00
Tonis Tiigi 56e1f71e6b cache: allow pushing exported cache 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-13 15:37:55 -07:00
Tonis Tiigi 1b3f191b37 cache: base code for cache importing 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-12 17:53:28 -07:00
Tonis Tiigi 8e611643f7 cache: export cache to content store 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-12 15:04:33 -07:00
Tonis Tiigi 0fd670fb73 solver: allow access to cached references for exporting 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-12 11:23:42 -07:00
Akihiro Suda 0ee59e2bdf Merge pull request #137 from tonistiigi/contentmask 
solver: replace contentkey with contentmask
 2017-10-07 09:47:38 +09:00
Tonis Tiigi 2716524dee exclude selector from content based input hash 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-06 15:43:48 -07:00
Tonis Tiigi 53018a8113 solver: replace contentkey with contentmask 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-05 19:58:05 -07:00
Tonis Tiigi 59910481ca solver: make nested build use solverequest 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-05 17:51:04 -07:00
Tonis Tiigi 23a114a977 solver: simplify solver public api 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-05 10:05:38 -07:00
Tonis Tiigi 80bc5f6097 fix exporter attributes interface returns 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-03 21:14:13 -07:00
Tonis Tiigi 2809d01cf6 frontend: add gateway frontend / image frontend support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-10-03 21:14:03 -07:00
Akihiro Suda f89ffc2ab4 llb: per-vertex metadata (e.g. IgnoreCache) 
cache invalidation itself it not implemented yet

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-10-03 05:36:24 +00:00
Tonis Tiigi 4a9142223d solver: fix possible data races 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-27 22:38:54 -07:00
Tonis Tiigi f41283caed solver: fix shared request progress and cancellation 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-27 12:45:47 -07:00
Tonis Tiigi 05c39612d8 dockerfile: resolve to digest ref in frontend 
Less hacky than keeping the cached resolver.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-22 15:49:37 -07:00
Tonis Tiigi d258bcab7b dockerfile: clean up converter 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-22 10:49:19 -07:00
Tonis Tiigi cc1b4140af bgfunc: safer locking 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-19 12:29:29 -07:00
Tonis Tiigi 01aaf130e5 solver: refactor to a shared vertex solver model 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-18 13:58:33 -07:00
Tonis Tiigi fd0f7ee2a8 frontend: fix dockerfile frontend resolver 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-15 18:09:28 -07:00
Tonis Tiigi 4c22f6c25a frontend: share attributes to exporter 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-15 18:09:28 -07:00
Akihiro Suda a26eba7cda fix for read-only mount 
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
 2017-09-14 06:42:12 +00:00
Tonis Tiigi 6d335c3c86 imagesource: share config resolver with frontend 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-05 13:11:27 -07:00
Tonis Tiigi 4040029810 dockerfile: improve copy implementation 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-09-01 19:27:54 -07:00
Tonis Tiigi 2d290db465 frontend: base of dockerfile frontend 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-08-31 23:27:25 -07:00
Tonis Tiigi eb7b5bb76a contenthash: use selector for content path 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-08-21 17:07:24 -07:00
Tonis Tiigi 8162e15787 llb: add support for mounting subpath or reference 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-08-07 18:45:50 -07:00
Tonis Tiigi 9f6d9a9e78 exporter: add local exporter 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-08-04 17:12:19 -07:00
Tonis Tiigi 8738929b8c solver: implement content based cache support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-08-04 10:03:26 -07:00
Akihiro Suda 39b9dec52f Merge pull request #87 from tonistiigi/snapshot-metadata 
cache: add more metadata to snapshots
 2017-07-27 09:36:10 +09:00
Tõnis Tiigi 5ecab5e142 Merge pull request #11 from dmcgowan/containerd-worker 
Containerd worker
 2017-07-26 14:36:51 -07:00
Tonis Tiigi 38ac00090c cache: add more metadata values to snapshots 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-26 11:28:20 -07:00
Derek McGowan 9ce5053a8b Update containerd version 
Fix for logrus rename, use fork until fixed in moby.
Removed unused tar stream.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
 2017-07-26 11:07:45 -07:00
Tonis Tiigi 93757aef7f nested llb builds support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-24 18:09:16 -07:00
Tonis Tiigi 2e89d2dc72 llb: add custom types for input indexes 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-21 16:39:25 -07:00
Tonis Tiigi 8b09e89dbb solver: avoid removing references required for future cache 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-20 17:05:37 -07:00
Tonis Tiigi 97b8395272 solver: always cache the original reference instance 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-19 11:08:49 -07:00
Tonis Tiigi ff951eecd9 llb: add readonly mounts support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-18 16:07:37 -07:00
Tonis Tiigi eac79f7c7e cache: refactor reference reuse and caching 
Replaces previous mutable.Freeze logic with
commits that can live together with mutable data.
Finalize method is added if the implementation
needs to make sure that the immutable ref is
flushed to the driver. Refs are automaitcally
finalized when writable layers are created on
top of them.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-18 16:01:01 -07:00
Tonis Tiigi 5c2e675d18 source: add local file source fundamentals 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-12 23:23:20 -07:00
Tonis Tiigi 29d912fddf exporter: fix exporter progress reporting 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-10 23:29:21 -07:00
Tonis Tiigi 45bfdd5e42 exporter: add image exporter 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-10 23:29:14 -07:00
Tonis Tiigi 2be8499bdc llb: add scratch support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-10 22:19:05 -07:00
Tonis Tiigi fa7ee2e2a8 source: add keepgitdir option for git source 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-10 22:19:05 -07:00
Tonis Tiigi b65989a5e4 source: add git source 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-10 22:19:05 -07:00
Tonis Tiigi bcb950915b llb: add const for root mount 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-10 21:54:38 -07:00
Tonis Tiigi 2bb4c5d487 worker: change mounts map to slice 
Fixes random mount order.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-10 19:18:19 -07:00
Tonis Tiigi 25ba9d72de solver: first pass of instruction cache 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-06 20:42:17 -07:00
Tonis Tiigi c9e2493f06 source: update interface for cache calls 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-06 12:12:17 -07:00
Tonis Tiigi f0c14879c7 solver: remove llb dependency 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-05 22:42:33 -07:00
Akihiro Suda 6f7452ddf1 Merge pull request #49 from tonistiigi/du-update 
Fixes for du view and releasing
 2017-07-03 16:09:11 +09:00
Tonis Tiigi daefcdc9f7 solver: move ops to separate files 
Preparation for making these unit testable.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-02 21:26:57 -07:00
Tonis Tiigi 246ce5e037 cache: fix reference releasing 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-07-01 08:37:30 -07:00
Tonis Tiigi f9bd9b5cb3 solver: fix vertex info becoming lost on progress replay 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-29 22:42:00 -07:00
Tonis Tiigi 65c2f8ef4c solver: cache vertex progress for replays 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-29 22:42:00 -07:00
Tonis Tiigi 2df68a5706 solver: add error reporting and canceling support 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-29 15:31:08 -07:00
Tonis Tiigi ec5c60283e solver: add vertex sharing on concurrent builds 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-28 21:52:05 -07:00
Tonis Tiigi b573060eea project: remove poc label 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-23 09:37:36 -07:00
Tonis Tiigi 728de510f3 llb: improve llb generation 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-21 15:21:36 -07:00
Tonis Tiigi 3d3190573c client: save logs to trace file 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-19 14:32:51 -07:00
Tonis Tiigi 0a7a2c1da3 solver: update exec ops to separate functions 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-19 14:32:50 -07:00
Tonis Tiigi 4ed7fa3280 progress: refactor progress IDs to items 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-19 14:32:50 -07:00
Tonis Tiigi 13f3a45b1d progress: refactor multireader and add status display 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-16 16:10:42 -07:00
Tonis Tiigi 4b41158252 solver: control vertexstatus for pull 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-15 16:08:45 -07:00
Tonis Tiigi 177aac4018 api: fix types in protobuf 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-13 23:35:35 -07:00
Tonis Tiigi 12a84368df solver: vertex progress reporting 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-13 23:35:29 -07:00
Tonis Tiigi 62b7d04d01 control: base of status reporting 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-13 22:52:59 -07:00
Derek McGowan 94d62fd432 Add context to functions which call snapshotter 
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
 2017-06-13 16:10:31 -07:00
Tonis Tiigi 84a14f44ed llb: ops execution example 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-09 14:49:01 -07:00
Tonis Tiigi bf8f1ee37a control: add simple solve method 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-09 14:49:01 -07:00
Tonis Tiigi d90d3cc683 solver: move client to separate pkg 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-09 14:49:01 -07:00
Tonis Tiigi 8719505435 solver: add release 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-09 14:49:00 -07:00
Tonis Tiigi 25bc7e02fc solver: base implementation 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2017-06-09 14:48:44 -07:00