890c3f77b6
Allow ExitError type to be transmitted over GRPC
...
This will allow clients to retrieve exit error codes returned during a
solve without parsing the error messages.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-07-28 15:01:05 -07:00
f4a67d7e9c
Exporter config digest typo
...
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-07-15 20:42:36 +02:00
18b49fd7dc
refactor to use util/bklog instead of using logurs directly
...
Signed-off-by: Morlay <morlay.null@gmail.com>
2021-07-13 11:42:31 +08:00
b055d2d55c
Merge pull request #2057 from ktock/export-compression
...
exporter: Enable to specify the compression type for all layers of the finally exported image
2021-07-06 21:52:36 -07:00
3152eab58f
Enable to forcefully specify compression type
...
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2021-07-07 12:00:06 +09:00
1a7543a105
Merge pull request #2163 from tonistiigi/detect-otel-env
2021-07-03 01:01:34 +09:00
8d70a9e098
Merge pull request #2215 from tonistiigi/improve-errors
...
Improve build errors and remove useless grpc prefix
2021-07-01 14:33:40 +09:00
a27d7c3d81
exec: improve error message on exec errors
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-30 21:10:37 -07:00
be8ab28a0a
Merge pull request #2081 from tonistiigi/local-differ-none
...
Allow none differ on local source to avoid false Dockerfile matches
2021-07-01 09:42:24 +09:00
ee39f5dd68
dockerfile: use none differ for dockerfile/dockerignore
...
This avoids wrong metadata matches on small files
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-29 16:17:08 -07:00
6eab36ddca
Merge pull request #2197 from aaronlehmann/hash-empty
...
Avoid nil pointer dereference when copying from image with no layers
2021-06-27 20:25:59 -07:00
a018bf5cd9
Test with tonistiigi/test:nolayers as well
...
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-06-27 16:09:42 -07:00
8f2fd48a6d
Check that scratch is mounted as empty dir
...
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-06-23 14:15:06 -07:00
e70f8556e5
Add test for copying from scratch
...
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-06-23 13:36:36 -07:00
01b50a3df1
llb: ensure meta resolver uses platform form constraints
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-22 21:49:07 -07:00
e0124e192c
llb: add constraints to async llb
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-22 21:34:43 -07:00
b4935eb79a
llb: add constraints to vertex and validate
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-22 18:47:32 -07:00
14d606c289
client: pass delegated exporter as parameter
...
Avoid client package having dependency on global detect package.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-20 21:57:41 -07:00
d512920c24
support collecting traces from llb.Exec
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-20 21:57:35 -07:00
d8fc4e15f9
enable collecting traces via control api
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-20 21:57:35 -07:00
7489cc8be0
vendor: update opentelemetry to 1.0.0-rc
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-19 10:02:26 -07:00
c773a215c1
integration: add common context base to all integration tests
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-18 20:49:49 -07:00
b10f25944b
correctly validate span from context
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-15 21:11:27 -07:00
8f50bae3f1
replace opentracing with opentelemetry tracers
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-15 21:11:27 -07:00
a7b155e30c
llb: don't allow content diff for now
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-11 21:15:10 -07:00
baa4fcdb0f
add differ support for local source
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-11 21:15:07 -07:00
8df5671844
Merge pull request #2082 from aaronlehmann/copy-include-exclude
...
Add IncludePatterns and ExcludePatterns options for Copy
2021-06-01 12:47:13 -07:00
cb536b26ca
Add test that non-included file does not invalidate cache
...
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-05-26 13:48:03 -07:00
b27fc6169b
Combine include/exclude caps into a single cap
...
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-05-26 13:48:03 -07:00
8e6fc83f16
Use require.ErrorIs
...
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-05-26 13:48:03 -07:00
b545c281c1
Add capabilities for copy include/exclude patterns
...
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-05-26 13:48:03 -07:00
7660af2f1d
Add IncludePatterns and ExcludePatterns options for Copy
...
Allow include and exclude patterns to be specified for the "copy" op,
similarly to "local".
Depends on https://github.com/tonistiigi/fsutil/pull/101
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-05-26 13:48:03 -07:00
a57ae2746f
Add test for handling relative mountpoints
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2021-05-26 11:22:16 -07:00
f113b49307
Merge pull request #2086 from thaJeztah/add_all_proxy
...
Add "ALL_PROXY" to list of default args / proxy env-vars
2021-04-26 09:09:44 -07:00
a743d4ba02
Add "ALL_PROXY" to list of default args / proxy env-vars
...
Relates to a82fff6377/docs/packages.md (proxies)
2021-04-24 18:19:02 +02:00
6ab9eeac79
Add regression test for run cache not considering mounts
...
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-04-14 08:53:00 -07:00
ca151bceaf
Move git protocol detection into seperate util
...
- the git protocol detection is required by buildx, and should reside in
a seperate exported gitutil package.
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
2021-03-31 12:42:57 -07:00
072126f710
Bump stargz snapshotter to v0.4.1
...
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2021-02-24 12:09:14 +09:00
8d70777537
fileop: fix checksum to contain indexes of inputs
...
Cache mismatch can happen if fileop switches input indexes
between different actions.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-03 22:17:57 -08:00
08e901325b
Merge pull request #1901 from alexcb/check-for-explicit-ssh-prefix
...
Check for explicit ssh git URL
2021-01-04 13:21:24 -08:00
3a5bdf0710
Move prefix check to isGitTransport
...
- rename IsSSHTransport to IsImplicitSSHTransport
- add testing for NewGitIdentifier
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
2021-01-04 11:09:56 -08:00
dd7fe19951
Merge pull request #1830 from coryb/unknown-exit-status
...
move UnknownExitStatus to executor package from errdefs
2020-12-28 16:35:14 -08:00
1098f8ee82
exporter: return config digest when exporting single-arch image
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-14 13:26:16 -08:00
bf5e780c5e
Merge pull request #1886 from tonistiigi/git-proto-fix
...
fix building from git url without a protocol
2020-12-10 23:04:15 -08:00
3fa063dcf3
move ssh regexp test to common location
...
- include testing
- improve hostname checking
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
2020-12-10 17:10:38 -08:00
23af9dcb86
require a username for git over ssh
...
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
2020-12-10 17:10:38 -08:00
0e441bedca
llb: make sure fileop uses the platform for current state
...
If source root sets the platform for a state, vertexes should
keep that instead of using the global value passed on marshal.
Already worked properly for exec, but not for file.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-10 12:24:15 -08:00
a6baa678ea
fix building from git url without a protocol
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-09 18:40:34 -08:00
9d3f55c400
move ExitError from solver/errdefs to frontend/gateway/errdefs
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-12-07 01:09:59 +00:00
e46cfab9fb
fix invalid cancellation errors on pull
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-01 00:27:31 -08:00
87508b4f0a
client: temporarily skip testClientSlowRootfsRef for containerd
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-25 23:36:55 -08:00
3581c46b37
Merge pull request #1802 from alexcb/fix-git-url-parsing
...
Fix parsing ssh-based git sources
2020-11-16 20:12:31 -08:00
5e5f527e06
Merge pull request #1732 from hinshun/exec-error
...
Allow gateway exec-ing into a failed solve with an exec op
2020-11-16 19:13:11 -08:00
4bf1852a08
differentiate git protocol vs git over ssh protocol
...
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
2020-11-16 15:04:37 -08:00
3c73a22430
Add default ssh mount and keyscans when missing
...
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
2020-11-16 13:23:37 -08:00
1240dd7795
Return committed readonly inputs and actives in exec error in MountIDs
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-16 13:21:23 -08:00
11f2a61ae1
Fix parsing ssh-based git sources
...
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
2020-11-16 12:33:34 -08:00
4c0ca1719f
Fix container release not capturing closure of loop variable
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:58:49 -08:00
c33bcd61a2
Rename OutputIDs to MountIDs
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
bcff7baf60
Allow scratch mounts with gateway.RunContainer
...
- Plumb default worker by adding GetDefault() to frontend.WorkerInfos
- To avoid cyclic dependency, refactor frontend.WorkerInfos to worker.Infos
- Refactor gateway.NewContainer to share code with llbsolver/ops/exec.go
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
bbc3f6df4c
Improve file action client test by adding test matrix
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
7e1dc9bec1
Refactor to file action indexed outputs
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
7ce58c31ba
Plumb op metadata to recreate failed ops with gateway exec
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
92e89a3374
Fixes related to the PR
...
- back out changes to changing the git url
- fix gid
- ignore global ssh config option when specifying known hosts
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
2020-11-12 17:32:29 -08:00
f2c8eb177c
Support cloning from non-standard git repos
...
- fixes assumption that ssh git clones must be via the `git` user.
- allows passing the SSH_AUTH_SOCK from the client to GitSource
- allows passing a known_host entry for ssh
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
2020-11-10 15:55:17 -08:00
4011b6ef1c
Merge pull request #1754 from tonistiigi/log-clipping
...
progress: add log limits and clipping
2020-11-06 13:53:38 +09:00
2b025f623d
update runc used in integration tests so exec no longer panics
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-11-02 06:34:43 +00:00
fbe1346912
replace github.com/docker/docker/pkg/locker with moby/locker v1.0.1
...
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 09:25:05 +01:00
ffd4ab232e
gateway exec: add platform and worker constraints to NewContainer api
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-28 03:54:49 +00:00
7f0d13f9a5
Add test for second solve using rootfs of first solve that error'ed from CalcSlowCache
...
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-10-27 12:12:38 -07:00
51decbe06d
client: use containerd grpc message size defaults
...
Same defaults already used on buildkitd side
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-26 14:51:47 -07:00
cec6dae19d
dockerfile: add test for default shell and path
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-24 11:12:15 -07:00
ecf070a027
exec: use platform specific default path
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-23 15:30:23 -07:00
b464f1e315
update go-runc to use runc.ExitError for container exec status
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-20 06:44:29 +00:00
5eaecb905c
Merge pull request #1731 from coryb/issue-1714
...
add tty support for runc executor
2020-10-19 14:52:31 -07:00
4d265f5d22
refactor tty output tests to avoid flakey sleeps
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-17 06:52:06 +00:00
e308ef7874
add tty support for runc executor
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-16 17:36:56 +00:00
059dac1e05
tests: Adding tests for hostname specifying for building
...
Adding tests to for hostname specifying for image building
Cover: #1301
Signed-off-by: Lu Jingxiao <lujingxiao@huawei.com>
2020-10-14 15:36:47 +08:00
5e7ae230b2
Add hostname specifying for building
...
Fix : #1301
Signed-off-by: Lu Jingxiao <lujingxiao@huawei.com>
2020-10-14 14:45:36 +08:00
78f3e86dc1
Merge pull request #1627 from coryb/gateway-exec-proto
...
update gateway to add ability to run and exec into containers
2020-10-04 20:11:45 +09:00
2f0738c47a
Default Cwd to "/"
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-02 20:54:00 +00:00
d1a14ae768
use grpc typed error for ExitMessage error
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-02 07:36:45 +00:00
e76703c3f6
fix minor nits
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-02 01:58:58 +00:00
2943a08389
Merge pull request #1686 from tonistiigi/git-cap
...
client: avoid checking token cap on default case
2020-09-17 10:17:26 -07:00
6361c6bad2
Merge pull request #1666 from ktock/sgz-oci
...
Enable to use stargz snapshotter without spawning plugin process
2020-09-17 16:00:19 +09:00
6b145fa964
client: avoid checking token cap on default case
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-09-16 19:29:06 -07:00
5fcf6dd426
update gateway exec mount tests to verity ssh and secret mounts
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-09-17 00:13:06 +00:00
3cc8aa0649
move ExecOps mounting logic into new package so logic can be reused from gateway exec
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-09-15 07:01:42 +00:00
dc3d45de87
make GrpcClient implement gateway.client.Client
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-09-13 20:41:18 +00:00
efb36aa53c
add cap support check to gatewayClientForBuild
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-09-13 20:11:17 +00:00
516b9ec844
add exec tests with multiple mounts
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-09-13 07:23:53 +00:00
b18c08c4ab
fix tests after busybox update
...
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-09-11 23:24:55 -07:00
cf3d695cc2
Enable to run integration tests with stargz snapshotter
...
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-09-03 16:52:21 +09:00
355e937e15
update gateway to add ability to run and exec into containers
...
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-08-31 21:13:59 +00:00
76189201a8
Add integration test for containerd and stargz snapshotter
...
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-08-27 15:50:11 +09:00
3b418db323
llb: Cache DefinitionOp inputs to support memoization during Marshal
...
llb.State.Marshal uses a vertexCache map (keyed by the llb.Vertex interface
value) to memoize which vertexes it has already visited while marshalling.
However, before this change, llb.DefinitionOp was constructing new pointers for
each value in the return slice of Inputs(), which meant that each input had a
different key in the vertexCache (due to being different pointer values). This
meant no memoization actually occured. I noticed this while using
llb.DefinitionOp with a fairly large graph when my program crashed after using
>16GB of RSS during a call to llb.State.Marshal.
The fix here avoids changing vertexCache as that would impact many other
vertex implementations. Instead, just DefinitionOp.Inputs() is updated to cache
the pointers it creates and pass that cache to its descendents. This fix
resulted in the program that was previously crashing out-of-memory to run
without any perceivable increase in RSS.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-08-10 17:23:04 -07:00
279d686fec
Merge pull request #1567 from everpeace/fix-authority-header
...
Setting `Host` part of address url to `:authority` pseudo header
2020-08-06 15:43:00 -07:00
545532ab0e
Merge pull request #1475 from sipsma/lazy-ref
...
Add support for lazily-pulled blobs in cache manager.
2020-08-06 12:54:45 -07:00
8d5e9f8294
new connection helper: podman-container://<CONTAINER>
...
Similar to `docker-container://` but for Podman containers.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-08-06 15:38:16 +09:00
9fcd482211
client: fix connhelper context canceling
...
Dial context is canceled after connection is
established so can’t be used for a long running process.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-05 23:25:13 -07:00
Aaron Lehmann
CrazyMax
Morlay
Tõnis Tiigi
ktock
Akihiro Suda
Akihiro Suda
Edgar Lee
Sebastiaan van Stijn
Alex Couture-Beil
Cory Bennett
Lu Jingxiao
Erik Sipsma