Commit Graph

482 Commits (e99111112725e9d74232c044bedff0ea98ca229d)

Author SHA1 Message Date
Erik Sipsma 21d6b1258f Work around concurrent map write issue in tests.
The merge diff tests were using some common LLB state objects across
multiple test cases, which each have their own buildkit client and ran
in parallel. The LLB client library does not appear to currently work
with such use cases where LLB states are shared, with errors about
concurrent map writes to marshal state constraints occasionally being
hit during test runs.

Until that issue is addressed (or it's made explicit that the LLB client
library is not expected to work in such a way), this works around the
problem by not sharing states and instead using common funcs that return
distinct state instances.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2022-02-08 17:19:00 -08:00
Erik Sipsma 0566b9a345 Add support for progress groups.
This allows clients to specify that LLB states should be grouped in
progress output under a custom name. Status updates for all vertexes in
the group will show up under a single vertex in the output.

The intended use cases are for Dockerfile COPY's that use MergeOp as a
backend and for grouping some other internal vertexes during frontend
builds.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2022-02-08 11:27:49 -08:00
Tõnis Tiigi 17e01bac9a
Merge pull request #2579 from tonistiigi/secret-env
exec: allow mounting secret environment variables
2022-02-08 10:09:55 -08:00
Aaron Lehmann b92a6170cf Fix missing capabilities in FileOp
Marshal the constraints after the capabilities are added, not before.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-02-07 12:31:25 -08:00
Cory Bennett 559d079902 Allow signals to be sent to gateway exec containers
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2022-02-02 16:57:10 +00:00
Tonis Tiigi 30a0b02479 exec: allow mounting secret environment variables
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-01-31 11:22:57 -08:00
CrazyMax 39f6b4e739
Bump github.com/containerd/containerd to v1.6.0-rc.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-01-28 10:18:59 +01:00
Tonis Tiigi dc21885891 hack: enable more linters
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-01-19 12:20:30 -08:00
Erik Sipsma ce012ab70c remote cache: support arbitrary layers as results
This change enables inline cache to work as expected with MergeOp by
supporting a new type of result, DiffResult, which enables results to be
specified as a specific ordered set of layers inside an image.
Previously, results could only be specified with a singe layer index,
which meant that they had to start at the image's base layer and end at
that index. That meant that merge inputs couldn't be specified as they
are often a subset of the image layers that don't begin at the base.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2022-01-11 13:27:05 -08:00
Erik Sipsma 0ec885106f Add DiffOp test cases.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2022-01-06 11:05:51 -08:00
Erik Sipsma 8c1e411d01 Add DiffOp support to solver and client.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2022-01-06 11:05:51 -08:00
Akihiro Suda 15fb1145af
Merge pull request #2491 from ktock/overlayfs-redirect
Disable redirect_dir for avoiding incorrect diff
2022-01-05 14:03:04 +09:00
Kohei Tokunaga bc5cfe960b Disable redirect_dir for overlayfs snapshotter
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2021-12-24 11:58:44 +09:00
Tõnis Tiigi 539be17089
Merge pull request #2428 from tonistiigi/default-user-groups
executor: make sure supplementary groups are set for unset user
2021-12-14 17:02:09 -08:00
Tõnis Tiigi 76234fa1c7
Merge pull request #2498 from tonistiigi/warnings-updates
Updates to warnings handling
2021-12-14 17:01:46 -08:00
Tonis Tiigi 872518e334 update warning type definition
Detail is now an array and URL is a separate field.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-12-13 21:38:20 -08:00
Akihiro Suda 0dfc2aa421
Merge pull request #2518 from everpeace/fix-authority-header
Setting just servername to :authority pseudo header in client when using TLS.
2021-12-11 18:39:32 +09:00
Shingo Omura ccbf7f3386
Setting just servername to :authority pseudo header in client when using tls.
HTTP/2(RFC7540) defines :authority pseudo header includes the authority portion
of target URI but it must not include userinfo part (i.e. url.Host).

However, when TLS certificate specified, grpc-go requires it must match
with its servername specified for certificate validation.

Signed-off-by: Shingo Omura <everpeace@gmail.com>
2021-12-10 21:13:25 +09:00
Erik Sipsma 2bfad4b0dc Change integration.Test from a func to a interface
Using an interface instead of a func is more flexible while achieving
the same effect. It allows you to succintly define a large number of
test cases as structs, as is common in table-driven testing.

A helper func is added that converts the existing test funcs into the
interface, so the change is fairly seamless.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-12-09 20:35:48 -08:00
Erik Sipsma 0539181757 Update tests for consistency w/ dockerd.
When vendored into moby, the local exporter uses a user map that results
in all files being given at most 755 permissions. This change updates
the test to use permissions less than or equal to that to make tests
equivalent whether running w/ dockerd worker or any other type.

Additionally, add assertions that exported images also have the expected
contents, which helps ensures consistency between images created by
dockerd and those created by vanilla buildkit.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-12-06 19:18:16 -08:00
Tonis Tiigi 71316c6f29 split warning message into short and detail
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-12-02 18:17:35 -08:00
Tonis Tiigi 50963e289e dockerfile: show redirect parser warnings
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-30 21:58:37 -08:00
Tonis Tiigi 7ee783e90c add source mapping support to warnings
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-30 21:58:37 -08:00
Tonis Tiigi 6cad384e93 support setting warnings from frontends
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-24 00:19:43 -08:00
Erik Sipsma 43a74261b3 Remove symlink, not target, in FileOp.Rm.
Before this change, if the path provided to FileOp.Rm was a symlink then
the target of the symlink would be removed instead of the symlink
itself. Now, the symlink will be removed instead. However, any symlinks
present in the parent dirs of the specified path will still be resolved
before calling os.Remove; this change only results in the base of the
specified path not being followed.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-11-18 12:44:12 -08:00
Erik Sipsma d73e62f878 Add initial MergeOp implementation.
This consists of just the base MergeOp with support for merging LLB
results that include deletions using hardlinks as the efficient path
and copies as fallback.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-11-18 11:10:48 -08:00
Akihiro Suda d47b46cf2a
Merge pull request #2432 from crazy-max/gofmt
go fmt: add //go:build
2021-10-29 12:32:52 +09:00
Tõnis Tiigi b33357bcd2
Merge pull request #2430 from crazy-max/cgroup-parent
cgroup-parent support
2021-10-28 10:21:03 -07:00
CrazyMax 54b8ff2fc8
go fmt: add //go:build
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-28 13:26:43 +02:00
Tõnis Tiigi 4b86211bed
Merge pull request #2405 from ktock/cachecompression
Propagate compression options to the inline cache export
2021-10-27 20:58:53 -07:00
CrazyMax c82ef274bf
cgroup-parent support
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-27 23:50:20 +02:00
Tonis Tiigi e3ca502c09 client: improve checks for user test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-10-27 11:06:45 -07:00
Tonis Tiigi 58ecc5ce14 executor: make sure supplementary groups are set for unset user
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-10-27 11:03:37 -07:00
CrazyMax 7b171f6b6a
update to containerd v1.6.0-beta.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-26 22:48:51 +02:00
Kohei Tokunaga f9e0346b34 Propagate compression options to the inline cache export
Co-authored-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2021-10-22 15:43:32 +09:00
CrazyMax 954c4721f7
use bytes as given size for tmpfs mount
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-19 19:47:17 +02:00
Tõnis Tiigi 41c0a18147
Merge pull request #2411 from crazy-max/tpmfs-size
add size to tmpfs mounts
2021-10-18 13:26:44 -07:00
CrazyMax 87e1fa7ecb
add size to tmpfs mounts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-18 10:54:52 +02:00
Tonis Tiigi 1c51e87e16 client: allow setting custom dialer for session endpoint
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-10-17 20:21:28 -07:00
Andrey Smirnov a5d1cfc1e4
fix: provide only available capabilities to insecure environment
The problem this change is trying to fix are the environments where some
capabilities are already dropped, so they can't be granted to the
job with `--security=insecure`.

I know that probably fixed set of capabilities was implemented to
provide a stable build environment, but at the same time this breaks
environments with reduced capabilities.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-10-15 19:57:32 +03:00
CrazyMax 4e32ae5630
Add support for ulimit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-08 11:50:09 +02:00
CrazyMax 6094339341
Add support for shm size
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-01 16:07:29 +02:00
Tonis Tiigi 35fcb28a00 Clean up old TODOs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-09-14 22:28:08 -07:00
Tonis Tiigi 8b5c4d74ef exporter: support creating blobs with zstd compression
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-09-05 22:43:15 -07:00
Kohei Tokunaga f8d30d567e Add `estargz` compression type
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2021-08-24 14:10:09 +09:00
Cory Bennett 0b683eb1ff add missing ExtraHosts to gateway exec
Also adding tests for ExtraHosts and NetMode via gateway exec

Signed-off-by: Cory Bennett <cbennett@netflix.com>
2021-08-15 02:51:37 +00:00
coryb 2bd37c8a7d
Merge pull request #2286 from aaronlehmann/exiterror-over-grpc
Allow ExitError type to be transmitted over GRPC
2021-08-11 15:51:23 -07:00
Koichi Shiraishi 50ae0d8cf4
all: unify the go-digest package import alias to digest
Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
2021-08-11 08:29:12 +09:00
Koichi Shiraishi 671ae38fb7
all: unify the specs-go package import alias to ocispecs
ocispecs means "O"pen "C"ontainer "I"nitiative image-spec/"specs"-go/v1
                      opencontainers          /image-spec/specs-go/v1

Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
2021-08-11 08:29:09 +09:00
Cory Bennett b8255fbceb fix SecurityMode being dropped on gateway container Start
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2021-07-30 19:33:44 +00:00