JohnHammond
/
buildkit
mirror of https://github.com/JohnHammond/buildkit.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,026 Commits
7 Branches
72 Tags
27 MiB
Commit Graph

396 Commits (d2aa9f03024cdb2dfcc52482486c87a0071f374f)

Author SHA1 Message Date
Tonis Tiigi 86154daec1 solver: check for input index existence 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2021-04-07 22:02:42 -07:00
Sebastiaan van Stijn 292685db37
solve: remove duplicate filepath.Join 
I saw these, and there was no comment in the code explaining if there was a reason
for this; these were added in 2be999ba52, but couldn't
find a mention about these particular lines in the review comments, and I couldn't
think of a special reason for it ':-)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2021-03-29 13:24:55 +02:00
Tonis Tiigi bee77d81b9 solver: check for edge dependency before merging 
In some cases edges that depend on each other
can have identical cache keys. This happens for
example when empty layers are optimized out by
the differ between identical commands. We need to
detect this case and avoid merge.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2021-03-18 19:55:35 -07:00
Akihiro Suda 13ea18ad08
vendor containerd (requred for rootless overlayfs on kernel 5.11) 
Required for rootless overlayfs on kernel 5.11
https://github.com/containerd/containerd/pull/5076

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
 2021-03-10 13:51:27 +09:00
Akihiro Suda 19d31737fb
Merge pull request #1963 from tonistiigi/release-count-fixes 2021-02-19 08:22:54 +09:00
Tonis Tiigi 8d70777537 fileop: fix checksum to contain indexes of inputs 
Cache mismatch can happen if fileop switches input indexes
between different actions.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2021-02-03 22:17:57 -08:00
Tonis Tiigi 42fb2a8771 avoid double release of same ref released multiple times 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2021-02-03 10:53:19 -08:00
Tonis Tiigi b4b0ece384 make sure result refs returned in errors are clone 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2021-02-03 10:49:38 -08:00
Tonis Tiigi 3660e5f9c8 don't commit cache mounts on error 
Cache mount instances are shared between multiple vertextes/builds
so if one of the cloned instance gets committed reference count
will get corrupted as other parts of the code still see reference as
mountable.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2021-02-02 23:20:44 -08:00
Tonis Tiigi dae8f156da update qemu emulators 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2021-01-21 18:33:00 -08:00
Tõnis Tiigi dd7fe19951
Merge pull request #1830 from coryb/unknown-exit-status 
move UnknownExitStatus to executor package from errdefs
 2020-12-28 16:35:14 -08:00
Tonis Tiigi 3889ef509c allow nil return from frontend 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2020-12-14 17:19:41 -08:00
Tibor Vass bea286f94c
Merge pull request #1871 from tonistiigi/scheduler-fix-for-testing 
solver: fix slow-cache status tracking with preprocess
 2020-12-11 11:47:45 -08:00
Tonis Tiigi 2f247f0340 solver: fix slow-cache status tracking with preprocess 
Preprocess should not be taken into account when calculating
cache-slow state. But it should be used to detect if dep is complete
so preprocess runs before parent's exec function.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2020-12-10 22:57:30 -08:00
Tonis Tiigi 1b271e0014 solver: delay before discarding job 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2020-12-09 20:53:03 -08:00
Cory Bennett 9d3f55c400 move ExitError from solver/errdefs to frontend/gateway/errdefs 
Signed-off-by: Cory Bennett <cbennett@netflix.com>
 2020-12-07 01:09:59 +00:00
Tõnis Tiigi d5f179bb79
Merge pull request #1863 from tonistiigi/solver-slowcache 
solver: improve slow cache detection logging
 2020-12-02 16:47:35 -08:00
Tonis Tiigi 706129170d solver: remove scheduler fix 
Asked in review.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2020-12-02 16:07:13 -08:00
Tonis Tiigi f38972a8ec mounts: handle optional ssh mount 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2020-12-02 14:28:51 -08:00
Tonis Tiigi 6a321dc574 solver: improve slow cache detection and add logging 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2020-12-01 22:38:15 -08:00
Tonis Tiigi e46cfab9fb fix invalid cancellation errors on pull 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2020-12-01 00:27:31 -08:00
Tonis Tiigi df3a9cad23 make sure execerror is released on cancellation 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2020-11-25 15:50:10 -08:00
Edgar Lee 372df78cc8 Guarantee err results are released when result proxy is released 
Signed-off-by: Edgar Lee <edgarl@netflix.com>
 2020-11-25 11:26:53 -08:00
Sebastiaan van Stijn 67bcfe699a
copy containerd.UnknownExitStatus to local const 
Copy this const to a local constant to prevent importing the containerd
client in the front-end.

For consistency, I also updated the executor code to use the same const,
although not strictly needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2020-11-18 00:11:09 +01:00
Tõnis Tiigi 5e5f527e06
Merge pull request #1732 from hinshun/exec-error 
Allow gateway exec-ing into a failed solve with an exec op
 2020-11-16 19:13:11 -08:00
Edgar Lee fa8a02c861 Fix optional cast for WithOp when unit testing 
Signed-off-by: Edgar Lee <edgarl@netflix.com>
 2020-11-16 13:37:30 -08:00
Edgar Lee 1240dd7795 Return committed readonly inputs and actives in exec error in MountIDs 
Signed-off-by: Edgar Lee <edgarl@netflix.com>
 2020-11-16 13:21:23 -08:00
Tõnis Tiigi 8dc3a54a78
Merge pull request #1782 from alexcb/acb-other-git-user 
SSH-based auth for llb.Git operations
 2020-11-16 10:53:44 -08:00
Edgar Lee 3ba6cd7bc2 Fix ExecError.EachRef invoking callback with possibly nil solver.Results 
Signed-off-by: Edgar Lee <edgarl@netflix.com>
 2020-11-13 23:20:41 -08:00
Edgar Lee c33bcd61a2 Rename OutputIDs to MountIDs 
Signed-off-by: Edgar Lee <edgarl@netflix.com>
 2020-11-13 22:05:13 -08:00
Edgar Lee bcff7baf60 Allow scratch mounts with gateway.RunContainer 
- Plumb default worker by adding GetDefault() to frontend.WorkerInfos
- To avoid cyclic dependency, refactor frontend.WorkerInfos to worker.Infos
- Refactor gateway.NewContainer to share code with llbsolver/ops/exec.go

Signed-off-by: Edgar Lee <edgarl@netflix.com>
 2020-11-13 22:05:13 -08:00
Edgar Lee a459eb4927 Return exec error for errors returned before executor 
Signed-off-by: Edgar Lee <edgarl@netflix.com>
 2020-11-13 22:05:13 -08:00
Edgar Lee 7e1dc9bec1 Refactor to file action indexed outputs 
Signed-off-by: Edgar Lee <edgarl@netflix.com>
 2020-11-13 22:05:13 -08:00
Edgar Lee 2d23d0cc43 Fix lint and unit tests for fileopsolver 
Signed-off-by: Edgar Lee <edgarl@netflix.com>
 2020-11-13 22:05:13 -08:00
Edgar Lee 7ce58c31ba Plumb op metadata to recreate failed ops with gateway exec 
Signed-off-by: Edgar Lee <edgarl@netflix.com>
 2020-11-13 22:05:13 -08:00
Tibor Vass fad612e748 Allow worker to override GetRemote(), needed in moby integration 
This fixes panics in the moby integration of buildkit when using cache.
Panics come from nil Differ in computeBlobChain which is called in GetRemote().

GetRemote() got moved from Worker to ImmutableRef during the lazy-pull refactor.
However, the ability to easily override GetRemote() got lost with that refactor.

This patch attempts to allow for the override while keeping changes minimal.

Signed-off-by: Tibor Vass <tibor@docker.com>
 2020-11-13 21:30:35 +00:00
Alex Couture-Beil f2c8eb177c Support cloning from non-standard git repos 
- fixes assumption that ssh git clones must be via the `git` user.
- allows passing the SSH_AUTH_SOCK from the client to GitSource
- allows passing a known_host entry for ssh

Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
 2020-11-10 15:55:17 -08:00
Tonis Tiigi e86356ddf8 solver: add input preprocess capability 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2020-11-03 14:49:44 -08:00
Tonis Tiigi bdcee17437 executor: change mount to struct 
Allows readonly passed cleanly.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2020-11-02 22:20:51 -08:00
Tonis Tiigi e3b05289d8 add session injection to remote loading 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2020-11-02 22:20:44 -08:00
Sebastiaan van Stijn fbe1346912
replace github.com/docker/docker/pkg/locker with moby/locker v1.0.1 
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
 2020-10-30 09:25:05 +01:00
Akihiro Suda 1b1d9e88dc
Merge pull request #1747 from tonistiigi/default-shell-path 
dockerfile: set correct default path and shell based on OS
 2020-10-26 16:37:12 +09:00
Tonis Tiigi cac0b68f9e solver: make sure job id cleared after build 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2020-10-25 21:50:40 -07:00
Tonis Tiigi ecf070a027 exec: use platform specific default path 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2020-10-23 15:30:23 -07:00
Edgar Lee cd42b669c9 Enable parallelization of NewContentHashFunc 
Signed-off-by: Edgar Lee <edgarl@netflix.com>
 2020-10-21 11:17:34 -07:00
Lu Jingxiao 5e7ae230b2 Add hostname specifying for building 
Fix: #1301

Signed-off-by: Lu Jingxiao <lujingxiao@huawei.com>
 2020-10-14 14:45:36 +08:00
Tonis Tiigi b4fad847ac frontend: support for subrequests 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2020-10-07 13:36:51 -07:00
Tonis Tiigi 44f27708b3 frontend: make sure inputs support for frontends is detectable from client 
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
 2020-10-04 10:44:16 -07:00
Cory Bennett 095a919c95 remove unnecessary MountManager interface 
Signed-off-by: Cory Bennett <cbennett@netflix.com>
 2020-10-02 06:24:34 +00:00
Cory Bennett 03e1c199a3 remove SessonManager from FrontendLLBBridge, adding as arg to Frontend.Solve 
Signed-off-by: Cory Bennett <cbennett@netflix.com>
 2020-10-02 01:59:05 +00:00