Commit Graph

3512 Commits (5872bf3dd16461a5d44e1e3d563418c594557782)

Author SHA1 Message Date
Akihiro Suda d71e905ad1
Merge pull request #2322 from thaJeztah/replace_toml
cmd/buildkitd: replace BurntSushi/toml with pelletier/go-toml
2021-08-24 01:15:32 +09:00
Aaron Lehmann 8021a3e667 Use fixed fileutils matching functions
This is important for two reasons:

1) Keeps caching logic consistent with recent fsutil changes to use
   these functions (also vendored here).

2) Allows us to move forward with removal of the original buggy Matches
   implementation in moby/moby.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-08-21 11:17:09 -07:00
Tõnis Tiigi 99368b8ecc
Merge pull request #2324 from tonistiigi/64bit-align-buildcount
control: fix 64bit alignment for buildcount
2021-08-20 12:16:44 -07:00
Tonis Tiigi 5cfe388b88 control: fix 64bit alignment for buildcount
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-08-20 10:39:19 -07:00
Sebastiaan van Stijn 0e6251510e
cmd/buildkitd: replace BurntSushi/toml with pelletier/go-toml
The BurntSushi/toml project has been deprecated, and the ecosystem
is converging on using pelletier/go-toml as the "canonical" replacement.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-20 18:57:04 +02:00
Aaron Lehmann 56c0981f1e Follow links in includedPaths to resolve incorrect caching when source path is behind symlink
As discussed in #2300, includedPaths does not resolve symlinks when
looking up the source path in the prefix tree. If the user requests a
path that involves symlinks (for example, /a/foo when a symlink /a -> /b
exists), includedPaths will not find it, and will expect nothing to be
copied. This does not match the actual copy behavior implemented in
fsutil, which will follow symlinks in prefix components of a given path,
so it can end up caching an empty result even though the copy will
produce a non-empty result, which is quite bad.

To fix this, use getFollowLinks to resolve the path before walking it.
In the wildcard case, this is done to the non-wildcard prefix of the
path (if any), which matches the behavior in fsutil.

Fixes the repro case here:
https://gist.github.com/aaronlehmann/64054c9a2cff0d27e200cc107bba3d69

Fixes #2300

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-08-18 11:54:48 -07:00
Akihiro Suda 7fa9e5778c
Merge pull request #2312 from tonistiigi/go1.17
update go to 1.17
2021-08-18 03:36:21 +09:00
Tonis Tiigi 2d18f3a923 gomod: update to go1.17
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-08-16 20:00:00 -07:00
Tonis Tiigi ac61da5890 update go to 1.17
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-08-16 19:56:45 -07:00
Tõnis Tiigi 124126e751
Merge pull request #2273 from sipsma/fix-2270
Fix flightcontrol test and usage bugs
2021-08-16 16:07:42 -07:00
Erik Sipsma b087e19e05 util: remove outdated flightcontrol test assertion.
The test was making an assertion that is no longer expected to always be
true after #2195, which purposely made flightcontrol less deterministic.
This lead to occasional failures.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-08-16 17:26:36 +00:00
Erik Sipsma 4237175ac7 pull: use resolvemode in flightcontrol key.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-08-16 17:26:36 +00:00
Erik Sipsma 808091a4e1 solver: include cachemap index in flightcontrol.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-08-16 17:26:36 +00:00
Erik Sipsma 23ec51611f cache: Fix flightcontrol use in computeBlobChain.
Previously, the flightcontrol group was being given a key just set to
the ref's ID, which meant that concurrent calls using different values
of compressionType, createIfNeeded and forceCompression would
incorrectly be de-duplicated.

The change here splits up the flightcontrol group into a few separate
calls and ensures that all the correct input variables are put into the
flightcontrol keys.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2021-08-16 17:26:36 +00:00
Tõnis Tiigi 7658c78597
Merge pull request #2294 from coryb/gateway-exec-extra-hosts
add missing ExtraHosts to gateway exec
2021-08-16 07:26:37 -07:00
Cory Bennett 2893203136 add gateway.exec.extrahosts capability
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2021-08-15 03:13:07 +00:00
Cory Bennett 0b683eb1ff add missing ExtraHosts to gateway exec
Also adding tests for ExtraHosts and NetMode via gateway exec

Signed-off-by: Cory Bennett <cbennett@netflix.com>
2021-08-15 02:51:37 +00:00
Akihiro Suda f6ac37d027
Merge pull request #2307 from thaJeztah/prepare_protobuf_update 2021-08-14 14:13:51 +09:00
Sebastiaan van Stijn e4e4bf3283
util/stack: update protoc options to work with newer versions
Generating the util/stack protos failed when updating protoc-gen-go to v1.5.2;
it looks like this is the only proto that's not generated using protoc-gen-gogo):

    util/stack/generate.go
    protoc-gen-go: unable to determine Go import path for "stack.proto"

    Please specify either:
        • a "go_package" option in the .proto source file, or
        • a "M" argument on the command line.

    See https://developers.google.com/protocol-buffers/docs/reference/go-generated#package for more information.

    --go_out: protoc-gen-go: Plugin failed with status code 1.
    util/stack/generate.go:3: running "protoc": exit status 1

Newer protobuf versions expect a go package to be set. Other .proto files in
this repository use the bare package name, but with protoc-gen-go v1.5.2, this
produces an error (package names must at least have a "/"). In addition to
including the option to the .proto file also changes the generated result
(`options go_package "<package name>"`).

Using the `-go_opt=M<package name>` option on the other hand, didn't change the
result (while still on protoc-gen-go v1.3.5), so I used that option instead.

protoc-gen-go v1.5.2 also changed the behavior where the generated file is stored,
seemingly relative to the `../../vendor` path specified. This coud be fixed either
by setting `--go_out=../../`, which was a bit counter-intuitive, or setting the
`--go_opt=paths=source_relative` option. The latter also prevented v1.5.2 from
storing the file in `utils/stack/github.com/moby/buildkit/utils/stack/` (sigh).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-13 22:19:10 +02:00
Tõnis Tiigi e8a3bb398f
Merge pull request #2305 from thaJeztah/generated_go_install
generated files: use "go install" to install binaries
2021-08-13 10:00:53 -07:00
Sebastiaan van Stijn bd4b02d5a8
generated files: use "go install" to install binaries
Now that this repository moved to go1.16, we can use 'go install' to install
these binaries.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-13 16:56:57 +02:00
Akihiro Suda 533a2024f0
Merge pull request #2304 from tonistiigi/required-fix
dockerfile: fix parsing required key without value
2021-08-13 04:16:52 +09:00
Tonis Tiigi 67352249e7 dockerfile: fix parsing required key without value
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-08-12 09:46:47 -07:00
coryb 2bd37c8a7d
Merge pull request #2286 from aaronlehmann/exiterror-over-grpc
Allow ExitError type to be transmitted over GRPC
2021-08-11 15:51:23 -07:00
Tõnis Tiigi 448dfad55e
Merge pull request #2301 from aaronlehmann/contenthash-tests
Additional tests and cleanup for cache/contenthash
2021-08-11 15:34:22 -07:00
Tõnis Tiigi 175e8415e3
Merge pull request #2277 from zchee/remove-dupimport
all: remove duplicate imports
2021-08-10 23:00:36 -07:00
Aaron Lehmann de785737db Fix IncludePattern/ExcludePattern matching
The transformation to rootedPatterns seems very wrong and inconsistent
with what the copy logic did. Change it to match the copy logic, and add
more testing.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-08-10 18:14:42 -07:00
Koichi Shiraishi 9e6ea26e39
golangci-lint: add go-digest importas setting
Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
2021-08-11 08:29:12 +09:00
Koichi Shiraishi 50ae0d8cf4
all: unify the go-digest package import alias to digest
Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
2021-08-11 08:29:12 +09:00
Koichi Shiraishi 414772d06e
golangci-lint: enable importas and add settings for specs-go package
Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
2021-08-11 08:29:12 +09:00
Koichi Shiraishi 9cc9d6c13b
hack/dockerfiles: upgrade golangci-lint version to v1.41.1
Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
2021-08-11 08:29:12 +09:00
Koichi Shiraishi 671ae38fb7
all: unify the specs-go package import alias to ocispecs
ocispecs means "O"pen "C"ontainer "I"nitiative image-spec/"specs"-go/v1
                      opencontainers          /image-spec/specs-go/v1

Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
2021-08-11 08:29:09 +09:00
Koichi Shiraishi 14a1d6e543
all: remove duplicate imports
Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
2021-08-11 08:28:30 +09:00
Aaron Lehmann b6ba966a68 Additional tests and cleanup for cache/contenthash
This adds a little extra testing around ** patterns, and adds a
(currently skipped) test for copying directories under symlinks (#2300).

It removes an extra call to `filepath.FromSlash` in `shouldIncludePath`
and an unused argument to that function.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-08-10 14:41:03 -07:00
Tõnis Tiigi 6034f46c3e
Merge pull request #2281 from thaJeztah/replace_deprecated_pkg_signal
replace use of deprecated github.com/docker/docker/pkg/signal
2021-08-10 10:50:47 -07:00
Tõnis Tiigi 3009c7b6da
Merge pull request #2289 from afbjorklund/socketmode
Set default socket permissions to 660
2021-08-10 10:49:46 -07:00
Tõnis Tiigi fd8823e70d
Merge pull request #2290 from coryb/gateway-exec-security-mode
fix SecurityMode being dropped on gateway container Start
2021-08-10 10:49:27 -07:00
CrazyMax fc3c182171
Merge pull request #2283 from thaJeztah/fix_invalid_import_constraints
util/tracing: remove incorrect import enforcing comment
2021-08-05 05:29:23 +02:00
Sebastiaan van Stijn cea791d3fe
replace use of deprecated github.com/docker/docker/pkg/signal
This package was moved to a separate module in github.com/moby/sys/signal

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-02 12:19:45 +02:00
Sebastiaan van Stijn 2e96deeba4
go.mod: golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c
In preparation of replacing the deprecated github.com/docker/docker/pkg/signal,
which uses this version (updating it separately for easier review).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-02 12:19:37 +02:00
CrazyMax 1a7c4807cd
Merge pull request #2292 from crazy-max/update-ctnd
bump containerd from 1.5.4 to 1.5.5
2021-08-02 00:41:02 +02:00
CrazyMax 55313cc41d
bump containerd from 1.5.4 to 1.5.5
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-08-01 15:08:33 +02:00
Cory Bennett b8255fbceb fix SecurityMode being dropped on gateway container Start
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2021-07-30 19:33:44 +00:00
CrazyMax e37dccd138
Merge pull request #2284 from crazy-max/fix-protoc-link
Fix protoc link
2021-07-30 18:04:16 +02:00
Anders F Björklund 75d6fc2589 Set default socket permissions to 660
The systemd default is 666, it seems.

Signed-off-by: Anders F Björklund <anders.f.bjorklund@gmail.com>
2021-07-30 13:41:14 +02:00
CrazyMax a83721aa6a
Merge pull request #2285 from crazy-max/runc-update
Update to github.com/opencontainers/runc v1.0.1
2021-07-29 14:38:32 +02:00
CrazyMax 8c64153810
Merge pull request #2287 from crazy-max/cache-docs
Split cache options doc for each exporter
2021-07-29 14:37:44 +02:00
CrazyMax 10e6e6acb0
Split cache options doc for each exporter
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-07-29 08:12:43 +02:00
CrazyMax c9fa27d017
Update to github.com/opencontainers/runc v1.0.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-07-29 00:42:51 +02:00
Aaron Lehmann 890c3f77b6 Allow ExitError type to be transmitted over GRPC
This will allow clients to retrieve exit error codes returned during a
solve without parsing the error messages.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-07-28 15:01:05 -07:00