Commit Graph

2751 Commits (7e1dc9bec143be0735b33c8f29ad69f6e6010592)

Author SHA1 Message Date
Edgar Lee 7e1dc9bec1 Refactor to file action indexed outputs
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
Edgar Lee 2d23d0cc43 Fix lint and unit tests for fileopsolver
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
Edgar Lee 7ce58c31ba Plumb op metadata to recreate failed ops with gateway exec
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
Tõnis Tiigi 6861f17f15
Merge pull request #1797 from tiborvass/fix-cache-moby-integration
Allow worker to override GetRemote(), needed in moby integration
2020-11-13 15:38:15 -08:00
Tibor Vass fad612e748 Allow worker to override GetRemote(), needed in moby integration
This fixes panics in the moby integration of buildkit when using cache.
Panics come from nil Differ in computeBlobChain which is called in GetRemote().

GetRemote() got moved from Worker to ImmutableRef during the lazy-pull refactor.
However, the ability to easily override GetRemote() got lost with that refactor.

This patch attempts to allow for the override while keeping changes minimal.

Signed-off-by: Tibor Vass <tibor@docker.com>
2020-11-13 21:30:35 +00:00
Tibor Vass ce283f39b5
Merge pull request #1773 from tonistiigi/preprocess
solver: add input preprocess capability
2020-11-12 21:07:00 -08:00
Tõnis Tiigi d168063ee5
Merge pull request #1796 from tiborvass/fixes_for_moby_vendoring
Fixes needed to vendor buildkit into moby
2020-11-12 21:05:32 -08:00
Tibor Vass 9dabe0545f Fixes needed to vendor buildkit into moby
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-11-13 01:51:55 +00:00
Tõnis Tiigi e68c2bc12f
Merge pull request #1789 from thaJeztah/bump_docker
vendor: docker, moby/sys/mount v0.1.1, moby/sys/mountinfo v0.4.0
2020-11-12 08:55:32 -08:00
Sebastiaan van Stijn f366587673
vendor: github.com/moby/sys/mountinfo v0.4.0
mountinfo v0.4.0
------------------------

Breaking changes:
 * `PidMountInfo` is now deprecated and will be removed before v1.0; users should
   switch to `GetMountsFromReader`

Fixes and improvements:
 * run filter after all fields are parsed
 * correct handling errors from `bufio.Scan`
 * documentation formatting fixes

mountinfo v0.3.1
------------------------

Breaking changes:
* `FstypeFilter` is changed to `FSTypeFilter` to be in-line with the field name.

Fixes and improvements:
* Fixed `FilterFunc` doc.

mountinfo v0.3.0
------------------------

Breaking changes:
* Some field names of `struct Mountinfo` are changed to be more in-line with Go
  naming conventions. Users need to be converted.

Fixes and improvements:
* Fixed per-package doc to be shown by `go doc`
* Make `GetMountsFromReader` Linux-specific.
* OpenBSD support added (same code as FreeBSD; not tested).

mountinfo v0.2.0
------------------------

Bug fixes:
* Fix path unescaping for paths with double quotes

Improvements:
* `Mounted`: speed up by adding fast paths using `openat2` (Linux-only, #29) and `stat`
* `Mounted`: relax path requirements (allow relative, non-cleaned paths, symlinks)
* Unescape `fstype` and `source` fields
* Documentation improvements

Testing/CI:
* Unit tests: exclude darwin
* CI: run tests under Fedora 32 to test openat2
* `TestGetMounts`: fix for Ubuntu build system
* Makefile: fix ignoring test failures
* CI: add cross build

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-12 13:21:44 +01:00
Sebastiaan van Stijn a72cda2d0e
vendor: github.com/moby/sys/mount v0.1.1
mount v0.1.1
------------------------
Improvements:
 - `RecursiveUnmount`: add a fast path
 - `Unmount`: improve doc
 - fix CI linter warning on Windows

Testing/CI:
 - Unit tests: exclude darwin
 - Makefile: fix ignoring test failures
 - CI: add cross build

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-12 13:21:13 +01:00
Sebastiaan van Stijn 9840fa21fc
vendor: docker/docker v20.10.0-beta1.0.20201110211921-af34b94a78a1
full diff: c2cc352355...af34b94a78

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-12 13:20:19 +01:00
Akihiro Suda bad541623d
Merge pull request #1781 from AkihiroSuda/arg-alpine-version
Dockerfile: use ARG for Alpine version
2020-11-12 12:54:50 +09:00
Akihiro Suda d2cea325e2
Dockerfile: use ARG for Alpine version
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-11-10 17:09:59 +09:00
Tõnis Tiigi 703a774918
Merge pull request #1780 from AkihiroSuda/remove-prebuilt-git
Dockerfile: remove pre-built git stage
2020-11-06 14:25:40 -08:00
Akihiro Suda 64123a45c3
Dockerfile: remove pre-built git stage
QEMU usermode emulation is robust enough for running `apk add --no-cache
git xz`, so can we can safely remove the pre-built git stage.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-11-06 16:15:11 +09:00
Tõnis Tiigi aba69fa102
Merge pull request #1779 from ktock/sgz-fuse
Add fuse dependency to moby/buildkit image
2020-11-05 23:09:01 -08:00
ktock 3182910bf4 Add fuse dependency to moby/buildkit image
Currently moby/buildkit image misses fuse dependency (fusermount) so currently
stargz support doesn't work on that image. This commit fixes this issue.

Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-11-06 15:08:36 +09:00
Akihiro Suda 4011b6ef1c
Merge pull request #1754 from tonistiigi/log-clipping
progress: add log limits and clipping
2020-11-06 13:53:38 +09:00
Tonis Tiigi 378fce2290 logs: show log limit in the clipping message
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-05 10:47:25 -08:00
Tõnis Tiigi cbbe079e2c
Merge pull request #1778 from AkihiroSuda/update-rootless-20201105
update RootlessKit, fuse-overlayfs, and containerd-fuse-overlayfs
2020-11-05 09:10:39 -08:00
Akihiro Suda b07668418d
update RootlessKit, fuse-overlayfs, and containerd-fuse-overlayfs
Changes:
- https://github.com/rootless-containers/rootlesskit/compare/v0.9.5...v0.11.0
- https://github.com/containers/fuse-overlayfs/compare/v1.1.2...v1.2.0
- https://github.com/AkihiroSuda/containerd-fuse-overlayfs/compare/v0.10.0...v1.0.0

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-11-05 17:22:56 +09:00
Tonis Tiigi e86356ddf8 solver: add input preprocess capability
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-03 14:49:44 -08:00
Tõnis Tiigi e48da241dc
Merge pull request #1761 from tonistiigi/session-remote
add session injection to remote ref loading
2020-11-03 08:33:28 -08:00
Tonis Tiigi bdcee17437 executor: change mount to struct
Allows readonly passed cleanly.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:51 -08:00
Tonis Tiigi e3b05289d8 add session injection to remote loading
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:44 -08:00
Tõnis Tiigi 35f41e9db4
Merge pull request #1775 from earthly/vlad/cni-bug
Fix race in CNI namespaces
2020-11-02 17:48:09 -08:00
Tõnis Tiigi febec7f76f
Merge pull request #1774 from coryb/rootless-exec
update runc used in integration tests so exec no longer panics
2020-11-02 12:02:37 -08:00
Vlad A. Ionescu 03987d2a75 Rename nativeID to nsPath. Simplify.
Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>
2020-11-02 11:39:17 -08:00
Vlad A. Ionescu 39fd96d2dd Fix race in CNI namespaces.
Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>
2020-11-02 11:23:28 -08:00
Cory Bennett 2b025f623d update runc used in integration tests so exec no longer panics
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-11-02 06:34:43 +00:00
Tibor Vass 2a6fcaeda9
Merge pull request #1769 from thaJeztah/bump_docker
vendor: github.com/docker/docker v20.10.0-beta1.0.20201030232932-c2cc352355d4
2020-10-31 02:03:07 -07:00
Sebastiaan van Stijn 0da7d8fdaa
vendor: github.com/docker/docker v20.10.0-beta1.0.20201030232932-c2cc352355d4
full diff: 4634ce647c...c2cc352355

also adds github.com/cilium/ebpf as a dependency, which I set to the same
version as is set in containerd/cgroups version

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-31 00:34:53 +01:00
Sebastiaan van Stijn 110ac24256
vendor: github.com/docker/cli v20.10.0-beta1.0.20201029214301-1d20b15adc38
full diff: 2298e6a3fe...1d20b15adc

relevant changes:
- config: don't call homedir on init()
- config: fix error message using incorrect filename
- config: remove redundant os.Stat()
- config: Handle errors on close in config file write.
- config: ignore empty config file instead of printing warning
- config: Fix ConfigFile.Save() replacing symlink with file
- Config-file: remove User-Agent from config.json when saving

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 20:45:20 +01:00
Sebastiaan van Stijn 1934fd7370
vendor: github.com/docker/libnetwork v0.8.0-dev.2.0.20200917202933-d0951081b35f
full diff: d8334ccdb9...d0951081b3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 20:45:18 +01:00
Sebastiaan van Stijn 72522d1740
vendor: golang.org/x/sys v0.0.0-20200922070232-aee5d888a860
full diff: 2334cc1a13...aee5d888a8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 20:45:15 +01:00
Sebastiaan van Stijn c3f78d3e8a
vendor: github.com/gorilla/mux v1.8.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 20:45:13 +01:00
Sebastiaan van Stijn 3d0c363f3f
vendor: github.com/Microsoft/go-winio v0.4.15-0.20200908182639-5b44b70ab3ab
full diff: fc70bd9a86...5b44b70ab3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 20:45:08 +01:00
Tõnis Tiigi cc27ed5212
Merge pull request #1766 from thaJeztah/integrate_dockerignore
Migrate github.com/docker/docker/builder/dockerignore to BuildKit repository
2020-10-30 12:04:24 -07:00
Tõnis Tiigi 8efe89b347
Merge pull request #1767 from thaJeztah/bump_shlex
vendor: github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510
2020-10-30 11:43:29 -07:00
Tõnis Tiigi 09299660eb
Merge pull request #1765 from ktock/sndoc
Add documentation about stargz/eStargz-based lazy pull functionality
2020-10-30 11:42:57 -07:00
Tõnis Tiigi 24f38cd342
Merge pull request #1764 from thaJeztah/replace_locker
replace github.com/docker/docker/pkg/locker with moby/locker v1.0.1
2020-10-30 11:41:59 -07:00
Akihiro Suda a4957ff806
Merge pull request #1762 from ktock/sgzconf
Delay decoding stargz config for avoiding extra dependency for config pkg
2020-10-30 21:03:50 +09:00
Sebastiaan van Stijn f9df470a4c
vendor: github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510
full diff: 6f45313302...e7afc7fbc5

- Remove unnecessary UnreadRune call
- Add go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 12:11:11 +01:00
Sebastiaan van Stijn 40e6129775
frontend/dockerfile: use dockerignore package from buildkit
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 11:55:05 +01:00
Sebastiaan van Stijn 31edeb3eab
dockerignore: remove import path enforcement comments
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 11:52:59 +01:00
Sebastiaan van Stijn a9580f569a
Migrate github.com/docker/docker/builder/dockerignore to BuildKit repository
Strategy taken:

    # install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
    brew install git-filter-repo

    cd ~/projects

    # create a temporary clone of docker
    git clone https://github.com/docker/docker.git docker_TEMP
    cd docker_TEMP

    # remove the origin remote (just for safety)
    git remote remove origin

    # create branch to work with
    git checkout -b migrate_dockerignore

    # remove all code, except for builder/dockerignore, and rewrite the files to frontend/dockerfile/dockerignore.
    git filter-repo --force --path builder/dockerignore --path-rename builder/dockerignore:frontend/dockerfile/dockerignore

    # check the results
    tree
    .
    └── frontend
        └── dockerfile
            └── dockerignore
                ├── dockerignore.go
                └── dockerignore_test.go

    3 directories, 2 files

    # go to the target github.com/moby/sys repository
    cd ~/projects/buildkit

    # create a branch to work with
    git checkout -b integrate_dockerignore

    # add the temporary repository as an upstream and make sure it's up-to-date
    git remote add docker_TEMP ~/projects/docker_TEMP
    git fetch docker_TEMP

    # merge the upstream code
    git merge --allow-unrelated-histories --signoff -S docker_TEMP/migrate_dockerignore

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 11:51:18 +01:00
ktock 5cbf8a152b Add documentation about stargz/eStargz support
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-10-30 19:25:41 +09:00
Sebastiaan van Stijn fbe1346912
replace github.com/docker/docker/pkg/locker with moby/locker v1.0.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 09:25:05 +01:00
ktock 0872ddf6aa Delay decoding stargz config for avoiding extra dependency for config pkg
`cmd/buildkitd/config` pkg is imported and used by other packages including
moby/moby.

Though stargz snapshotter configuration is currently effective only with
buildkitd + OCI worker, `cmd/buildkitd/config` consumer needs to introduce an
indirect dependency to stargz snapshotter's config pkg
(`github.com/containerd/stargz-snapshotter/stargz/config`), which is too much.

This commit solves this by delaying decoding the stargz config until OCI
worker's initialization phase.

Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-10-29 23:44:43 +09:00