Commit Graph

200 Commits (d21254e7f74b49a84c3fbe1b13260cb23954cf92)

Author SHA1 Message Date
Tonis Tiigi baa4fcdb0f add differ support for local source
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-11 21:15:07 -07:00
Tõnis Tiigi 8df5671844
Merge pull request #2082 from aaronlehmann/copy-include-exclude
Add IncludePatterns and ExcludePatterns options for Copy
2021-06-01 12:47:13 -07:00
Aaron Lehmann cb536b26ca Add test that non-included file does not invalidate cache
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-05-26 13:48:03 -07:00
Aaron Lehmann 8e6fc83f16 Use require.ErrorIs
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-05-26 13:48:03 -07:00
Aaron Lehmann 7660af2f1d Add IncludePatterns and ExcludePatterns options for Copy
Allow include and exclude patterns to be specified for the "copy" op,
similarly to "local".

Depends on https://github.com/tonistiigi/fsutil/pull/101

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-05-26 13:48:03 -07:00
Edgar Lee a57ae2746f Add test for handling relative mountpoints
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2021-05-26 11:22:16 -07:00
Tõnis Tiigi f113b49307
Merge pull request #2086 from thaJeztah/add_all_proxy
Add "ALL_PROXY" to list of default args / proxy env-vars
2021-04-26 09:09:44 -07:00
Sebastiaan van Stijn a743d4ba02
Add "ALL_PROXY" to list of default args / proxy env-vars
Relates to a82fff6377/docs/packages.md (proxies)

> (..) the first four of these are the standard built-in build-arg options
> available for `docker build`
> (..) The last, `all_proxy`, is a standard var used for socks proxying. Since
> it is not built into `docker build`, if you want to use it, you will need to
> add the following line to the dockerfile:
>
>     ARG all_proxy

Given the we support all other commonly known proxy env-vars by default, it makes
sense to add this one as well.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-24 18:19:02 +02:00
Aaron Lehmann 6ab9eeac79 Add regression test for run cache not considering mounts
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-04-14 08:53:00 -07:00
ktock 072126f710 Bump stargz snapshotter to v0.4.1
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2021-02-24 12:09:14 +09:00
Tonis Tiigi 8d70777537 fileop: fix checksum to contain indexes of inputs
Cache mismatch can happen if fileop switches input indexes
between different actions.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-03 22:17:57 -08:00
Tonis Tiigi 1098f8ee82 exporter: return config digest when exporting single-arch image
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-14 13:26:16 -08:00
Lu Jingxiao 059dac1e05 tests: Adding tests for hostname specifying for building
Adding tests to for hostname specifying for image building

Cover: #1301
Signed-off-by: Lu Jingxiao <lujingxiao@huawei.com>
2020-10-14 15:36:47 +08:00
Akihiro Suda 6361c6bad2
Merge pull request #1666 from ktock/sgz-oci
Enable to use stargz snapshotter without spawning plugin process
2020-09-17 16:00:19 +09:00
Tonis Tiigi b18c08c4ab fix tests after busybox update
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-09-11 23:24:55 -07:00
ktock cf3d695cc2 Enable to run integration tests with stargz snapshotter
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-09-03 16:52:21 +09:00
ktock 76189201a8 Add integration test for containerd and stargz snapshotter
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-08-27 15:50:11 +09:00
Erik Sipsma 55cbd19dec Add support for lazily-pulled blobs in cache manager.
This allows the layers of images to only be pulled if/once they are actually
required.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-08-05 17:18:43 -07:00
Paul "TBBle" Hampson 5e3e18a72d Use OS-agnostic os.SameFile instead of inode comparision
This fixes the build of this test under Windows.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-28 13:50:48 +10:00
Tõnis Tiigi a67af5b10c
Merge pull request #1581 from alexcb/tar-exporter-with-socket-copy-reproduce-error
fix socket handling during copy
2020-07-23 12:07:15 -07:00
Alex Couture-Beil 5382a2056e Treat unix sockets as regular files
This fix is similar to the fix in #1144; but was hit in a different code
path.

Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
2020-07-23 11:25:48 -07:00
Tonis Tiigi c8190b1518 lint: enable golint
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:24 -07:00
Paul "TBBle" Hampson 156d66da02 Revert "Pin containerd runtime to v1"
This reverts commit 9290c15ffc.

This was pinned during the upgrade to containerd 1.3 series, which
changed the default runtime on Linux to io.containerd.runc.v2.

No specific rationale was listed for this pinning, and clearly it's the
wrong thing to do in the presence of Windows, which does not have this
runtime.

Instead, we rely on the containerd-internal defaults, which distinguish
the runtimes for Linux and Windows.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-16 15:06:20 +10:00
Erik Sipsma 43b58b2016 client test: log skip in checkAllReleasable.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-06-24 12:07:00 -07:00
Erik Sipsma 9e0870415d Test media type after first push in TestBuildExportWithUncompressed
There is a bug in the way images are pushed that results in oci types being used
for layers even when docker types should be used, but only in single-layer
images.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-06-24 09:53:34 -07:00
Erik Sipsma 83af7c3c94 client test: Fix TestBuildExportWithUncompressed
This test had previously been accidentally not executing past the check for
whether containerd was running. A previous commit fixes that check, this commit
fixes a few bugs that had been going unnoticed in the containerd-specific part
of the test's code.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-06-21 20:42:53 -07:00
Erik Sipsma 463ec47ba0 client test: Fix check for whether sandbox has containerd
Before this, the check was always returning that containerd wasn't running and
thus skipping the rest of several test cases.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-06-21 20:42:53 -07:00
Edgar Lee fbee6cccbd Fix source map test in client_test
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-05-20 19:04:25 -07:00
Edgar Lee 7c81e16b8a Fix duplicate source maps and fix issue preventing multiple locations per source map
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-05-20 18:48:20 -07:00
Edgar Lee 59fa9fc9a0 Allow multiple source locations to be added as a constraint
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-05-20 16:48:44 -07:00
Tonis Tiigi 90c5e67496 client: add source mapping tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-19 17:22:14 -07:00
Tonis Tiigi 37b8832d00 upgrade errors checks to Is()
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-18 22:53:38 -07:00
Tonis Tiigi d64e417b1c hack: allow testing with dockerd
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-08 23:27:20 -07:00
Sam Whited c62983bbde all: run integration tests against dockerd
Signed-off-by: Sam Whited <sam@samwhited.com>
2020-04-08 23:27:17 -07:00
Tonis Tiigi 1a9d366b49 llb: asyncronous llb graph generation support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-03 14:55:10 -07:00
Xiaofan Zhang 0947ed31ea vendor: update fsutil to c2c7d7b0e144 and symlink test
Signed-off-by: Xiaofan Zhang <xiaofan.zhang@clinc.com>
2020-03-31 02:59:34 +00:00
Tonis Tiigi 09e8a06c04 client: improve cache key loop test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-03-02 22:35:59 -08:00
Anca Iordache 8c65b5bbd5 Tests build with local cache export
- to detect infinite loop caused by circular dependencies

Signed-off-by: Anca Iordache <anca.iordache@docker.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-03-02 22:35:59 -08:00
Edgar Lee c83e8bf2a9 Add gateway apicap for CapReferenceOutput and move reference llb.Output to method ToOutput()
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-01-23 11:36:30 -08:00
Edgar Lee 0a76749067 Add platform defop test, force renames in copy tests, and move forwarded solve test to dockerfile package
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-01-23 11:02:40 -08:00
Edgar Lee 2edb431a0f Allow previous solve results to be used in new solves
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-01-23 11:02:40 -08:00
Wei Fu 65388edc6c exporter: support compression for layer blob data
Allow user to choose the compression type for layer data. Gzip is
default compression for layer exporter, which consume more Cpu resources
and take long time to export. With compression option, user can use
nocompressed option to export to save time. And future, zstd is one new
option for end-user.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-12-09 23:51:38 +08:00
Edgar Lee c122adacb5 Implement file action remove for wildcards
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2019-10-30 14:25:00 -07:00
Tonis Tiigi fef0996045 dockerfile: skip cni tests on rootless
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-09-05 16:49:48 -07:00
Tonis Tiigi 1d7e7a6b51 llbsolver: fix error on multiple cache importers
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-29 15:21:17 -07:00
Tibor Vass 77ca4bce2e client: add exporter test for sockets
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-08-27 20:53:37 +00:00
Tonis Tiigi ac8fd77049 client: update tests to new deps
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-20 17:03:05 -07:00
Michael Crosby 9290c15ffc Pin containerd runtime to v1
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-20 17:03:04 -07:00
Tonis Tiigi 2cea1107d6 travis: add ci caching
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-17 09:13:06 -07:00
Tonis Tiigi fb3f2aeb63 client: add test for output target exists
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-31 10:21:26 -07:00
Tonis Tiigi 96b6a28312 exporter: allow oci exporters visibility to response metadata
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-31 10:21:19 -07:00
Tonis Tiigi 7b1bae7a42 solver: support no-cache for exec cache mounts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-26 14:42:32 -07:00
Akihiro Suda d750cb188a
Merge pull request #1073 from tonistiigi/cni-net
add cni networking support
2019-07-19 10:47:27 +09:00
Andrey Smirnov df52fc7f9c Skip test in rootless mode
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-07-18 22:50:53 +03:00
Andrey Smirnov fc9781a77c Add test for 'rw' sysfs/cgroup re-mount
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-07-18 18:53:27 +03:00
Tonis Tiigi e7759a861b client: add cni network tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-10 18:00:06 -07:00
Tonis Tiigi 1ef979a7bb llb: fix excessive formatting
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-01 16:55:08 -07:00
Tonis Tiigi 8e47580f71 fix flaky TestBasicInlineCacheImportExport
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-06-10 14:16:42 -07:00
Tõnis Tiigi 758c61e873
Merge pull request #570 from kunalkushwaha/entirlement-revised
security entitlement support
2019-03-28 16:14:06 -07:00
Tonis Tiigi 01a8de7d74 integration: secmode switch support and security testcase
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Kunal Kushwaha <kunal.kushwaha@gmail.com>
2019-03-27 13:57:03 +09:00
Kunal Kushwaha a2bbb5ff39 security entitlement support
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2019-03-27 13:57:03 +09:00
Tonis Tiigi 7e52b33bf8 integration: generalize pkg
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-21 23:49:08 -07:00
Tibor Vass 9a9e755588
Merge pull request #887 from tonistiigi/push-by-digest
exporter: allow push by digest in image exporter
2019-03-20 19:13:09 -07:00
Tonis Tiigi 8f174bcf52 client: add hardlinks local export test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-18 19:12:34 -07:00
Tonis Tiigi 37bfc6546c exporter: allow push by digest in image exporter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-18 16:00:48 -07:00
Tonis Tiigi 0d17ac323e fileop: updates with new fsutil copy pkg
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 17:49:58 -07:00
Tonis Tiigi 81a5fa5a2e llbsolver: fileop implementation
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-15 16:22:35 -07:00
Akihiro Suda 5c9f7b8ff0 buildctl: new CLI ("Option C+")
See https://github.com/moby/buildkit/pull/807#issuecomment-468146089

Close #774

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-03-06 13:20:21 +09:00
Akihiro Suda 46bde0d6b1 test: make sure `--export-cache type=inline` is not needed for import
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-02-23 21:01:27 +09:00
Tonis Tiigi 5434a42a6a client: add inline remote cache tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-01-21 19:39:15 -08:00
Akihiro Suda 6e7617e889 support local cache exporter and importer
Export:

  $ buildctl build ... --export-cache type=local,store=/path/to/output-dir

Import:

  $ buildctl build ... --import-cache type=local,store=/path/to/input-dir

Impact on CLI:
* Old (deprecated but still effective): `--export-cache localhost:5000/myrepo:buildcache --export-cache-opt mode=max`
* New: `--export-cache type=registry,ref=localhost:5000/myrepo:buildcache,mode=max`

Impact on API:
* New fields are added to control.proto and gateway.proto. The daemon
internally translates old API calls to the new ones.
* While new API can be used for `registry` caches, the client continues
to use the legacy API for `registry` caches to ensure compatibility with
old daemons.
* To import `local` caches with a frontend, the frontend needs to support
a new frontend opt `cache-imports`.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-01-19 02:38:26 +09:00
Tonis Tiigi 1f51f5ea11 testutil: avoid excessive daemons in integration tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-11 16:47:35 -07:00
Tibor Vass c9b01072d0 solver: specify SSH key ID in error message when required key was not forwarded
Signed-off-by: Tibor Vass <tibor@docker.com>
2018-10-05 22:50:07 +00:00
Tonis Tiigi cd451ec063 executor: make sure hostname is resolvable
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-02 23:24:37 -07:00
Tonis Tiigi a75983ddb9 integration: pass mirrored images from test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-18 22:07:38 -07:00
Tonis Tiigi f7120867f1 client: add regression test for closed stdin
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-11 14:26:46 -07:00
Tonis Tiigi 347420e77f sshprovider: allow keys from local files
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-10 14:12:41 -07:00
Tonis Tiigi 9337126e6e client: add ssh socket test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-07 08:57:11 -07:00
Tõnis Tiigi 2759005f6a
Merge pull request #575 from ijc/frontend-set-export-name
exporters: allow frontend control of the exported image name
2018-08-31 10:27:09 -07:00
Ian Campbell 03a9cb89af exporter: Use "*" to signify that client allows frontend to set image name
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-31 17:34:49 +01:00
Ian Campbell 0b20bcc8f3 The OCI image exporter does not support naming, so reject such attempts.
Adjust the test case to not name in the OCI case.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-31 14:23:34 +01:00
Ian Campbell aa51b40649 Include exported image name in ExporterResponse metadata
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-31 11:07:32 +01:00
Tõnis Tiigi c9711fcc26
Merge pull request #574 from ijc/return-frontend-metadata
Allow some Metadata to be return from frontend to the control client
2018-08-30 10:08:51 -07:00
Ian Campbell 755bf5387e Moved some subtests into `t.Run` blocks.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-30 16:28:19 +01:00
Ian Campbell 3db7441e06 clarify naming in testFrontendImageNaming
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-30 15:59:12 +01:00
Ian Campbell 8d114533c2 exports: require caller to request frontend-based naming, by passing name of `%s`.
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-30 15:59:06 +01:00
Akihiro Suda decb27618b containerd: set timeout explicitly (60s)
The contained default value was shortened from 60s to 10s in containerd/containerd#2554

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-08-23 18:14:57 +09:00
Ian Campbell fa39d134ee exporters: allow frontend control of the exported image name
Returning a metadata item named "image.name" will set the name to use.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-17 13:20:36 +01:00
Ian Campbell fdf13cf96e Allow some Metadata to be return from frontend to the control client
Propagate anything in the `frontend.*` namespace from the `frontend.Result`
into the `client.SolveResponse`.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-17 12:45:21 +01:00
Tonis Tiigi 130f5f5ab0 solver: net host with basic entitlements support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-09 14:03:35 -07:00
Tonis Tiigi bf29f5bd3b client: add test for extra hosts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-03 13:56:13 -07:00
Tonis Tiigi 97fa2b527a cache: add all option to prune
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-26 22:55:05 -07:00
Tonis Tiigi f719aadcba dockerfile: add tests for multi-platform exports
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 16:01:12 -07:00
Tonis Tiigi 53f657a5db client: add secret mounts test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-19 22:02:11 -07:00
Tonis Tiigi 3fa70c6b25 client: simplify parallel sources test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-09 16:27:08 -07:00
Ian Campbell 0dd25df7e2 client: add test case for multiple parallel builds.
It appears that multiple parallel builds suffer some cross talk between their
mount/snapshots e.g. this new case currently fails because iterations see files
in `/src/` which correspond to other iterations (i.e. iteration 1 might see a
file called "test2.txt"). The `ls -i` and `stat` output also confirms that
where tests see the same filename they are also apparently seeing the same
inode.

Note that each solve uses a `SolverOpt.SharedKey` of the (unique) source
directory so there should be no clashes. Use of `llb.sharedKeyHint()` is
avoided since using a unique one of these hides the issue (AIUI uniqueness of
`sharedKeyHint` is only required with a session, not between them).

The shell fragment within the test is a bit crazy, it tries to output as much
info about what it saw vs what was expected to aid debugging.

I have also been seeing cases in my own code where `cacheContext.checksum`
fails the `not found` test at the top, which seems like either the wrong thing
is mounted or it has gone away or otherwise been mutated unexpectedly. I had
been hoping that this test case would also reproduce this, but so far it has
not.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-07-09 15:44:28 -07:00
Derek McGowan 618f34a1f9 Support creating and pushing OCI types
Adds image and oci exporter option "oci-mediatypes"
Ensures that the images created in the content store
have the correct type which matches the manifest.
Sets the correct media type on the descriptor in push from
reading the type specified in the manifest.
Removes use of distribution manifest packages.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-06-28 11:23:35 -07:00
Tonis Tiigi ccbf185006 llbsolver: allow sharing modes for cache mounts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-19 15:57:20 -07:00
Ian Campbell 97ffd6cce8 client: take a context.Context to New().
This allows two things:

- The caller to set a shorter timeout than previously hardcoded 30s. In
  `buildctl` reduce the timeout to 5s. Since the existing timeout has gone
  callers will need to arrange to pass one themselves.
- The caller can arrange for the context to be cancelled for other reasons, use
  this in `buildctl` to plumb through the Ctrl-C handling, meaning that
  `buildctl` now exits almost immediately on Ctrl-C instead of after several
  seconds.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-06-11 14:28:08 +01:00
Tonis Tiigi 30461204da vendor: update fsutil to fix local source bug
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-10 09:22:59 -07:00