22f632f895
allow frontends to be run directly on the control client side
...
This allows builder code to be written which can be built as either a gateway
container or in a purely client side configuration, giving implementors more
flexibility.
Now when `Solve` sees a request with neither a definition nor a frontend
specified it will make the job available via new LLBBridge endpoints on the
control socket which the client can then use. These end points require the job
id to be present in the gRPC metadata and a client side object is added to
facilitate this.
The `llbBridgeForwarder` type is now exposed as a public `interface
LLBBridgeForwarder` which satisfies the underlying gRPC server interface
(`pb.LLBBridgeServer`) as well as a new `Done()` & `Result()` pair which can be
used to wait for the client to call `Return()` (using a model similar to
`context.Context`).
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-14 11:50:08 +01:00
95e0348f57
dockerfile: enable global net-mode settings
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-09 14:03:40 -07:00
130f5f5ab0
solver: net host with basic entitlements support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-09 14:03:35 -07:00
785436a312
Merge pull request #563 from ijc/progressui-custom-operation
...
progressui: allow caller to customise "Building" string
2018-08-09 10:49:29 -07:00
809b9a4602
Merge pull request #568 from tonistiigi/multi-platform
...
dockerfile: allow multi-target external builds
2018-08-10 00:09:35 +09:00
a6e6f3fd09
Merge pull request #567 from tonistiigi/dockerfile-secrets
...
dockerfile: implement secret type mounts
2018-08-10 00:09:17 +09:00
2a4ec0b192
gateway: call `Return` if call to `BuildFunc` returns an error.
...
Reorder the code slightly so that an error returned by the call to the
`BuildFunc` is picked up by the `defer`d error handling.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-09 11:19:42 +01:00
13f53d5753
gateway: tolerate `Solve` being called with no `creq.Definition`
...
Avoids panicing when accessing `creq.Definition.Metadata`.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-09 11:19:42 +01:00
3f5276b3c4
gateway: make `Return` a one shot operation and check for various edge cases.
...
Refactor the setting/getting of the result for a `llbBridgeForwarder` in order
to check that `Return` is only called once and only with the correct options.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-09 11:19:42 +01:00
5485309658
gateway: Return an `interface` from `grpcclient.New`.
...
Returning an interface rather than a private-struct from a public interface is
good practice.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-09 11:19:29 +01:00
fa83e55b21
dockerfile: allow multi-target external builds
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-08 18:42:43 -07:00
5e6729e56f
dockerfile: implement secret type mounts
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-08 18:12:16 -07:00
de1c0cc8de
Merge pull request #556 from tonistiigi/netproviders
...
executor: allow network providers
2018-08-09 08:46:04 +09:00
ccc559bc43
progressui: allow caller to customise "Building" string
...
In clients which are doinging multiple builds or phases it can be useful to say
something more specific here (e.g. "Building first image", "Probing" etc)
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-08 16:29:01 +01:00
682dce615a
gateway: support direct creation/running of a client.
...
Provide a `New` method and a new `Run` method on the `grpcClient` to allow
manual creation of a client in addition to the current ability to initialise
from the environment. Accordingly the existing `Run` method becomes
`RunFromEnvironment`.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-08 10:17:32 +01:00
f8dd602282
runc: improve canceling
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-07 11:51:56 -07:00
765f1b64b9
executor: allow network providers
...
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-07 11:51:56 -07:00
47c4888ab1
Merge pull request #551 from tonistiigi/gc-prune
...
cache: add gc style pruning support
2018-08-07 11:14:46 -07:00
9f85386315
Merge pull request #561 from trusch/feature/multiple-tags-while-exporting
...
exporters: containerimage: allow targetName to contain multiple values
2018-08-07 11:13:47 -07:00
f9ab15123d
exporters: containerimage: allow targetName to be a comma separated list and loop through it while pushing.
2018-08-07 09:59:47 +02:00
1a37ee3bf4
cache: add gc style pruning support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-06 21:45:39 -07:00
c6aa1817a1
Merge pull request #558 from ijc/exporters-multiple-custom-metadata
...
exporters: allow for more than one non-standard metadata key
2018-08-06 11:11:45 -07:00
e37916545d
Merge pull request #557 from ijc/exporter-nil-map
...
Initialise `src.Metadata` in all `{image,oci}ExporterInstance.Export`
2018-08-06 11:10:46 -07:00
6757e8dfe9
Merge pull request #553 from tonistiigi/add-hosts
...
llbsolver: add support for extra host records
2018-08-06 11:08:29 -07:00
075c54589c
exporters: allow for more than one non-standard metadata key
...
I think this was just a thinko.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-06 10:23:46 +01:00
fb13b26419
Initialise `src.Metadata` in all `{image,oci}ExporterInstance.Export`
...
Failure to do so would potentially lead to a panic. Belt & braces followup
for #555 .
`localExporterInstance` has no `e.meta` to merge and makes no use of the
`Metadata` anyhow (since that doesn't seem to make sense for a local export) so
is unmodified.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-06 09:59:00 +01:00
96f24ca7bb
executor: improve hosts cleanup
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-03 14:01:54 -07:00
f8b0573edb
dockerfile: expose add-hosts
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-03 13:56:14 -07:00
bf29f5bd3b
client: add test for extra hosts
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-03 13:56:13 -07:00
f7359f7093
llb: add extrahost support
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-03 13:56:13 -07:00
4945fe758c
llbsolver: add support for extra host records
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-03 13:56:13 -07:00
7e98ece83d
Merge pull request #555 from ijc/exporter-panic
...
solver: avoid dereferencing nil res.Metadata
2018-08-03 13:15:52 -07:00
f7937f1989
solver: avoid dereferencing nil res.Metadata
...
I saw this:
panic: assignment to entry in nil map
goroutine 3173 [running]:
github.com/moby/buildkit/exporter/oci.(*imageExporterInstance).Export(0xc42094ac40, 0xecac60, 0xc4211ca690, 0xed12c0, 0xc42000fca8, 0x0, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/moby/buildkit/exporter/oci/export.go:113 +0x156
github.com/moby/buildkit/solver/llbsolver.(*Solver).Solve.func2(0xecac60, 0xc4211ca690, 0xc4202207c0, 0x0)
/go/src/github.com/moby/buildkit/solver/llbsolver/solver.go:132 +0x7d
github.com/moby/buildkit/solver/llbsolver.inVertexContext(0xecac60, 0xc4211ca690, 0xe2ec38, 0x1d, 0xc420d43400, 0x0, 0x0)
/go/src/github.com/moby/buildkit/solver/llbsolver/solver.go:203 +0x1f6
github.com/moby/buildkit/solver/llbsolver.(*Solver).Solve(0xc4203f7dc0, 0xecac60, 0xc420deb830, 0xc4203ed200, 0x19, 0xc4202ff840, 0x0, 0x0, 0x0, 0x0, ...)
/go/src/github.com/moby/buildkit/solver/llbsolver/solver.go:131 +0x761
github.com/moby/buildkit/control.(*Controller).Solve(0xc42017e870, 0xecac60, 0xc420deb830, 0xc4201edd40, 0xc42017e870, 0x1, 0x1)
/go/src/github.com/moby/buildkit/control/control.go:207 +0x4b8
github.com/moby/buildkit/api/services/control._Control_Solve_Handler.func1(0xecac60, 0xc420deb800, 0xde2180, 0xc4201edd40, 0xecac60, 0xc420deb800, 0xed67a0, 0x1595288)
/go/src/github.com/moby/buildkit/api/services/control/control.pb.go:810 +0x86
github.com/moby/buildkit/vendor/github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1(0xecac60, 0xc420deb800, 0xde2180, 0xc4201edd40, 0xc4202ff9e0, 0xc4202ffa00, 0x0, 0x0, 0xebbea0, 0xc420188310)
/go/src/github.com/moby/buildkit/vendor/github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc/server.go:57 +0x2ba
main.unaryInterceptor.func1(0xecaba0, 0xc4206afe40, 0xde2180, 0xc4201edd40, 0xc4202ff9e0, 0xc4202ffa00, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/moby/buildkit/cmd/buildkitd/main.go:330 +0x15f
github.com/moby/buildkit/api/services/control._Control_Solve_Handler(0xd76a00, 0xc42017e870, 0xecac60, 0xc420deb530, 0xc42028d030, 0xc420450760, 0x0, 0x0, 0x34, 0x3)
/go/src/github.com/moby/buildkit/api/services/control/control.pb.go:812 +0x167
github.com/moby/buildkit/vendor/google.golang.org/grpc.(*Server).processUnaryRPC(0xc420244700, 0xed39c0, 0xc420694000, 0xc420671680, 0xc420497650, 0x152cef8, 0x0, 0x0, 0x0)
/go/src/github.com/moby/buildkit/vendor/google.golang.org/grpc/server.go:1011 +0x4fc
github.com/moby/buildkit/vendor/google.golang.org/grpc.(*Server).handleStream(0xc420244700, 0xed39c0, 0xc420694000, 0xc420671680, 0x0)
/go/src/github.com/moby/buildkit/vendor/google.golang.org/grpc/server.go:1249 +0x1318
github.com/moby/buildkit/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc42063e0f0, 0xc420244700, 0xed39c0, 0xc420694000, 0xc420671680)
/go/src/github.com/moby/buildkit/vendor/google.golang.org/grpc/server.go:680 +0x9f
created by github.com/moby/buildkit/vendor/google.golang.org/grpc.(*Server).serveStreams.func1
/go/src/github.com/moby/buildkit/vendor/google.golang.org/grpc/server.go:678 +0xa1
Which was due to `res, err := s.Bridge(j).Solve(ctx, req)` having `res.Metadata
== nil`. There are several paths in `llbBridge.Solve()` where this can be the
case, plus a case where this comes from a frontend which should not be allowed
to crash the daemon.
Likely introduced by d70d816dee6be1257f5d
2018-08-03 16:08:04 +01:00
e57eed420c
Merge pull request #549 from tonistiigi/dockerfile-names
...
dockerfile: custom names to vertexes
2018-07-31 10:58:56 -07:00
04a5cc81e8
dockerfile: add platform prefixes
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-31 10:41:36 -07:00
d0aafaefd0
dockerfile: add stage prefixes
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-31 10:41:36 -07:00
4c44d2c6e4
dockerfile: custom names to vertexes
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-31 10:41:30 -07:00
a5f442f2f4
Merge pull request #546 from tonistiigi/config-progress
...
llbsolver: show status of resolving image config
2018-07-30 12:06:35 +09:00
f668043366
Merge pull request #545 from tonistiigi/shared-refs-detect
...
cache: detect refs shared with image store
2018-07-30 12:05:53 +09:00
b52c0002db
llbsolver: show status of resolving image config
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-27 15:59:11 -07:00
fe3388e70d
solver: change builder.call to builder.context
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-27 15:39:14 -07:00
d1ad6961e3
cache: detect refs shared with image store
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-27 14:08:50 -07:00
3910818f92
Merge pull request #544 from tonistiigi/cache-record-type
...
cache: add record type field to usage record
2018-07-28 05:12:07 +09:00
97fa2b527a
cache: add all option to prune
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-26 22:55:05 -07:00
489246dd28
cache: support for internal/frontend record type
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-26 22:54:53 -07:00
57b96a0ee5
cache: add record type field to usage record
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-26 22:54:39 -07:00
aa81807f1c
Merge pull request #542 from tonistiigi/resolvemode
...
llb: add resolvemode to image source
2018-07-26 14:04:27 -07:00
4159e2c45a
Merge pull request #543 from tonistiigi/filtering
...
Add filtering support for diskusage and prune
2018-07-26 10:32:37 -07:00
87f43efda5
cache: implementation for prune filtering
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 19:35:49 -07:00
b966096ab5
dockerfile: allow setting image resolve mode
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 18:23:59 -07:00
Ian Campbell
Tonis Tiigi
Akihiro Suda
Kunal Kushwaha
Tino Rusch
Tibor Vass