Commit Graph

535 Commits (master)

Author SHA1 Message Date
Tõnis Tiigi be424a99ea
Merge pull request #1728 from earthly/vlad/authorizer-errors
Prevent context canceled errors from being permanent in authorizer
2020-10-15 17:46:14 -07:00
Vlad A. Ionescu d127edf990 Prevent context canceled errors from being permanent in authorizer
Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>

Check if context is canceled before returning cached token or err.

Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>

Fix possible race conditions

Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>

Fix block

Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>
2020-10-15 15:11:59 -07:00
Tõnis Tiigi 4177e7ca28
Merge pull request #1660 from tonistiigi/token-fetch
auth: fetch tokens from client side
2020-10-12 16:13:19 -07:00
Tonis Tiigi b4fad847ac frontend: support for subrequests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-07 13:36:51 -07:00
Tonis Tiigi 44f27708b3 frontend: make sure inputs support for frontends is detectable from client
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-04 10:44:16 -07:00
Akihiro Suda 6361c6bad2
Merge pull request #1666 from ktock/sgz-oci
Enable to use stargz snapshotter without spawning plugin process
2020-09-17 16:00:19 +09:00
Tonis Tiigi 1f94445456 auth: fetch tokens from client side
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-09-15 13:40:30 -07:00
Ilya Dmitrichenko 6e8fde8431
vendor: update containerd to d4e7820
Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
2020-09-07 10:40:09 +01:00
ktock cf3d695cc2 Enable to run integration tests with stargz snapshotter
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-09-03 16:52:21 +09:00
Erik Sipsma a57f8a2dcc
Merge pull request #1402 from ktock/remote-snapshotter
Support stargz snapshotter for dev stages
2020-08-31 10:41:11 -07:00
Tonis Tiigi d33756e17b vendor: update containerd to efa0e809
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-30 23:04:25 -07:00
ktock 76189201a8 Add integration test for containerd and stargz snapshotter
Signed-off-by: ktock <ktokunaga.mail@gmail.com>
2020-08-27 15:50:11 +09:00
Miguel Ángel Jimeno 6e1d460d4c
cni: remove duplicate error check
Signed-off-by: Miguel Ángel Jimeno <miguelangel4b@gmail.com>
2020-08-23 21:51:01 +02:00
Jon Zeolla 4f966cc3da Fix platform typo
Signed-off-by: Jon Zeolla <zeolla@gmail.com>
2020-08-17 19:31:02 -04:00
Tonis Tiigi e6500927d2 resolver: handle nil group properly
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-13 17:27:25 -07:00
Tonis Tiigi e322304c07 resolver: avoid retrying fatal error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-13 17:27:25 -07:00
Tonis Tiigi 8f8dccf837 resolver: add docs to new functions
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-13 17:27:25 -07:00
Tonis Tiigi 5f69909037 resolver: switch post/get order on requesting token
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-13 17:27:25 -07:00
Tonis Tiigi b474dbf55f resolver: clean up unused resolver pool
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-13 17:27:18 -07:00
Tonis Tiigi 21c4ab8621 resolver: fix token expiring issues
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-13 17:27:18 -07:00
Tonis Tiigi f1cd79bf65 resolver: add better pooling and custom authenticator
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-13 17:27:11 -07:00
Tõnis Tiigi 545532ab0e
Merge pull request #1475 from sipsma/lazy-ref
Add support for lazily-pulled blobs in cache manager.
2020-08-06 12:54:45 -07:00
Tonis Tiigi 862387445e grpcerrors: use full typeurl registration
Also switches current types to json.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-06 12:28:48 -07:00
Erik Sipsma 926ca1804c pull: de-pointer non-optional platform field
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-08-05 17:18:43 -07:00
Erik Sipsma cdcf49fd18 Fix progress in schema1 pull.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-08-05 17:18:43 -07:00
Erik Sipsma 55cbd19dec Add support for lazily-pulled blobs in cache manager.
This allows the layers of images to only be pulled if/once they are actually
required.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-08-05 17:18:43 -07:00
Tonis Tiigi 22061b1ec9 resolver: reenable keep-alive
Was disabled with http2 but shouldn’t have been.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-02 09:32:46 -07:00
Tonis Tiigi 5e08a24e88 resolver: make sure authorizer is not regenerated
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-02 09:32:40 -07:00
Tonis Tiigi 3278f8d6d7 pull: fix counter for ensuremanifestloaded check
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-01 22:08:22 -07:00
Tonis Tiigi cdf424f677 resolver: enable tracing on default transport
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-29 23:19:52 -07:00
Akihiro Suda 594ebbfac0
Merge pull request #1601 from tonistiigi/insecure-fix
push: reenable setting insecure exporter opt
2020-07-29 13:55:36 +09:00
Tonis Tiigi 942c39adfd add testing for containerd 1.4-beta2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-28 18:13:15 -07:00
Paul "TBBle" Hampson e48eaf9619 Log HostComputeNamespace name instead of the whole object
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-28 18:07:20 +10:00
Paul "TBBle" Hampson 0c9d862645 Fix undefined variable in Windows-specific test
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-28 18:07:20 +10:00
Paul "TBBle" Hampson b5e6d54910 Add errors-on-non-Linux ToRootless implementation
`go test` complains about this tree because the package ends up with no
symbols on non-Linux platforms, e.g., Windows, even though this API is
never be called on non-Linux platforms, according to the compiler.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-28 18:07:20 +10:00
Paul "TBBle" Hampson f617b0828a Fix testutil/integration build on Windows
Although we don't run any integration tests on Windows at this time,
this does mean that the SKIP_INTEGRATION_TESTS env-var works, so we can
run a wider range of non-integration tests without spurious failures.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-28 13:50:48 +10:00
Tonis Tiigi 846142df61 push: reenable setting insecure exporter opt
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-26 23:27:19 -07:00
Tonis Tiigi 67044875c4 resolver: allow setting both insecure tls and http
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-26 18:29:45 -07:00
Tonis Tiigi 9cb0654ce4 integration: lock buffer for concurrent writes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-23 12:04:14 -07:00
Paul "TBBle" Hampson 50c5d4ce51 Don't add loopback CNI configuration on Windows
There's no Loopback CNI plugin for Windows as far as I can see, so I'm
assuming it's not needed.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-21 16:31:32 +10:00
Paul "TBBle" Hampson 1fedff0e4b Reorder CNI config to set options before consumers
The `WithLoNetwork` and `WithConfFile` options consume the
`WithPluginDir` and `WithInterfacePrefix` settings, so they must come
first.

This fixes the issue that the provided plugin directory was not used for
the provided configuration file. This would have been more visible, but
the default `InterfacePrefix` is already "eth".

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-21 16:31:31 +10:00
Paul "TBBle" Hampson dc2574ceae Implement separate handling for Windows Network namespaces
The Windows API is completely different, so the common code cannot
create the unique identifier, but must accept what the OS-specific code
does.

For Linux, it's a pathname. In Windows, it's a GUID string.

This pulls in direct usage of Microsoft/hcsshim, which previously was
only an indirect usage via containerd calls from, e.g., localMounter on
Windows.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-21 16:31:31 +10:00
Paul "TBBle" Hampson 1645a1e2ca Provide separate NetProviders map for Windows
This enforces the lack of host networking support on Windows.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-21 16:31:31 +10:00
Tonis Tiigi 94c202bfb9 handle error returns more properly
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:36 -07:00
Tonis Tiigi 5da4a40ae8 lint: enable more linters
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:36 -07:00
Tonis Tiigi 06c4197c05 rename binfmt_misc to archutil
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:36 -07:00
Tonis Tiigi c8190b1518 lint: enable golint
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-19 09:28:24 -07:00
Tonis Tiigi 57a88b0afb lint: add misspell and unused
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-18 08:13:57 -07:00
Tonis Tiigi 04233f90b3 replace gometalinter with golangci linter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-18 08:13:48 -07:00
Tonis Tiigi 55e29c1abe integration: fix rootless worker binaries closing after tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-17 19:30:40 -07:00
Tonis Tiigi 8247b7b50b integration: increase timeouts
Set the limit high enough to ensure failure in CI
signals a bug

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-17 19:30:40 -07:00
Tonis Tiigi adb68c2764 integration: add concurrent sandbox limit
Default parallelization seems off in CI

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-17 19:30:33 -07:00
Tõnis Tiigi 919ee62a78
Merge pull request #1574 from sipsma/fix-containerd-test
Use new containerd shim in integ tests; don't skip on requirement error.
2020-07-16 11:26:45 -07:00
Erik Sipsma f828f65516 Use new containerd shim in integ tests; don't skip on require error.
A recent change updated the containerd shim built for integ tests to the newer
v2, but the integ test code itself was still hardcoded to look for the older
shim binary and, when that binary wasn't found, skip running the test. This
caused all containerd tests to be skipped.

The fix here updates the binary name to v2 and also removes the branch that
will skip a test when the requirements check fails.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-07-16 10:21:21 -07:00
Paul "TBBle" Hampson 7369864344 Move printLogs from oci.go to sandbox.go
It's not OCI-specific, so it makes more sense in the file that exports
it to the world.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-17 02:35:16 +10:00
Tonis Tiigi a8c2137598 resolver: add credentials cache
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-05 12:07:21 -07:00
Tonis Tiigi 05013a663b pull: fix session updating on resolver
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-05 00:15:33 -07:00
Tonis Tiigi 214aa5dbcf pull: allow separate sessions for different parts of pull
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-05 00:15:33 -07:00
Tonis Tiigi 2e9987ad16 session: track sessions with a group construct
Avoid hidden session passing and allow one session to drop when
multiple builds share a vertex.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-05 00:15:25 -07:00
Akihiro Suda c25f2217dc
Merge pull request #1548 from tonistiigi/push-fix
push: dedupe push handlers
2020-06-30 12:49:37 +09:00
Tonis Tiigi bd29ad98cd progressui: fix logs time formatting
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-06-29 14:39:41 -07:00
Tonis Tiigi 375bfa60a2 contentutil: synchronize writers backed by pusher
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-06-29 12:40:05 -07:00
Tonis Tiigi d8e8fda57c push: dedupe push handlers
This allows to work around the containerd issue on
tracking push status on pushing manifest lists.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-06-29 12:08:18 -07:00
Erik Sipsma 463ec47ba0 client test: Fix check for whether sandbox has containerd
Before this, the check was always returning that containerd wasn't running and
thus skipping the rest of several test cases.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-06-21 20:42:53 -07:00
Sebastiaan van Stijn 932c39505b
use containerd/sys to detect UserNamespaces
The implementation in libcontainer/system is quite complicated,
and we only use it to detect if user-namespaces are enabled.

In addition, the implementation in containerd uses a sync.Once,
so that detection (and reading/parsing `/proc/self/uid_map`) is
only performed once.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-06-15 12:51:20 +02:00
Tonis Tiigi cc88b8f4f4 increase test context timeouts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-06-06 15:48:55 -07:00
Tonis Tiigi 6073e6cff3 llb: enable source tracking
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-18 15:50:03 -07:00
Sebastiaan van Stijn 3ddd1fc23c
Fix goimports linting
```
[5/5] RUN --mount=target=/go/src/github.com/moby/buildkit 	gometalinter ...
0.435 util/rootless/specconv/specconv_linux.go:1:⚠️ file is not goimported (goimports)
1.320 cache/manager.go:1:⚠️ file is not goimported (goimports)
1.335 cache/manager_test.go:1:⚠️ file is not goimported (goimports)
1.337 cache/migrate_v2.go:1:⚠️ file is not goimported (goimports)
1.342 cache/refs.go:1:⚠️ file is not goimported (goimports)
1.454 cache/remotecache/registry/registry.go:1:⚠️ file is not goimported (goimports)
2.285 cmd/buildctl/build.go:1:⚠️ file is not goimported (goimports)
3.082 executor/oci/user.go:1:⚠️ file is not goimported (goimports)
4.333 session/content/content_test.go:1:⚠️ file is not goimported (goimports)
4.614 snapshot/containerd/content.go:1:⚠️ file is not goimported (goimports)
4.721 solver/errdefs/vertex.go:1:⚠️ file is not goimported (goimports)
6.066 util/network/cniprovider/cni.go:1:⚠️ file is not goimported (goimports)
ERROR: executor failed running [/bin/sh -c gometalinter --config=gometalinter.json ./...]: buildkit-runc did not terminate successfully
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-13 17:38:56 +02:00
Tonis Tiigi a82c3c0728 progressui: fix panic on terminals with zero height
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-11 22:35:06 -07:00
Tibor Vass 10889212c4
Merge pull request #1454 from tonistiigi/errdefs
errdefs: add support for typed errors
2020-04-27 15:57:55 -07:00
Simon Ferquel 6d3f568629 Avoid creation of irrelevant temporary files on Windows
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
2020-04-27 15:44:57 +02:00
Tonis Tiigi 90288ab716 errdefs: update to new packages
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-24 11:25:44 -07:00
Tonis Tiigi c511cdb35a util: add grpcerrors and stack package
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-22 15:58:20 -07:00
Tonis Tiigi 37b8832d00 upgrade errors checks to Is()
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-18 22:53:38 -07:00
Tonis Tiigi 54c55e6aa9 entitlements: separate daemon and client errors
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-15 15:28:31 -07:00
Akihiro Suda ec5d112053
Merge pull request #1435 from thaJeztah/default_no_trunc
progress: disable truncating by default when using --progress=plain
2020-04-16 02:54:22 +09:00
Sebastiaan van Stijn 75ab2b1137
progress: disable truncating by default when using --progress=plain
Currently, `docker build --progress=plain` truncates output, which makes it
difficult to debug problems during build (the step that failed may not be the
cause of the failure).

For example, in the following build, the output of the `RUN echo ...` is truncated:

    DOCKER_BUILDKIT=1 docker build --no-cache --progress=plain -t foo -<<EOF
    FROM busybox
    RUN echo "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas ut lorem nec leo euismod vestibulum. Donec tortor nisl, iaculis at vehicula vel, interdum eu orci. Integer velit lacus, congue id magna eu, mollis accumsan augue. Aliquam non venenatis risus, eu posuere libero. Vestibulum ante ipsum primis in faucibus orci luctus." > /somewhere
    RUN echo "something went wrong"; exit 1
    EOF

    #5 [2/3] RUN echo "Lorem ipsum dolor sit amet, consectetur adipiscing elit....
    #5 DONE 0.2s

    #6 [3/3] RUN echo "something went wrong"; exit 1
    #6 0.211 something went wrong
    #6 ERROR: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully
    ------
     > [3/3] RUN echo "something went wrong"; exit 1:
    ------
    failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully

While there is an existing `PROGRESS_NO_TRUNC` environment variable, I think that
this should be the default if the user opted to use `--progress=plain` (or in
situations where no TTY is attached, which could be in CI).

This patch changes the default to disable truncating in those situations, but
allowing users to opt-out by setting `PROGRESS_NO_TRUNC=0`

With this change the same build looks like this:

    #5 [2/3] RUN echo "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas ut lorem nec leo euismod vestibulum. Donec tortor nisl, iaculis at vehicula vel, interdum eu orci. Integer velit lacus, congue id magna eu, mollis accumsan augue. Aliquam non venenatis risus, eu posuere libero. Vestibulum ante ipsum primis in faucibus orci luctus." > /somewhere
    #5 sha256:b2f0f47d63911ee55e7cf17c81007310e28190b5be84aa1a7869ba90786d5cee
    #5 DONE 0.2s

    #6 [3/3] RUN echo "something went wrong"; exit 1
    #6 sha256:c037b34bb998ae7d30572b489286da14df87e1478adf6d0c5c71c79b84b11bcc
    #6 0.293 something went wrong
    #6 ERROR: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully
    ------
     > [3/3] RUN echo "something went wrong"; exit 1:
    ------
    failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-09 17:52:01 +02:00
Tonis Tiigi d64e417b1c hack: allow testing with dockerd
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-08 23:27:20 -07:00
Sam Whited c62983bbde all: run integration tests against dockerd
Signed-off-by: Sam Whited <sam@samwhited.com>
2020-04-08 23:27:17 -07:00
Wei Fu 9f063f6b61 resolver: disable http2 for pushing
The golang net/http package uses http2 client to serve https by default,
if let Transport.TLSNextProto is nil. And net/http package doesn't
provide tunnable value for http2 flow control which will limit push
performance.

Before this commit, use GODEBUG="http2debug=1" buildkitd to pushing
one image from dockerfile like

```
$ about 700MB
FROM scratch
ADD ./golang-1.13.0-stretch.tar.gzip /
```

and use ifstat to monitor network interface and found that

```
$ ifstat -i enp0s3
      enp0s3
 KB/s in  KB/s out
    0.47      0.67
    0.44      0.51
   19.72     11.56
   62.25   2184.41
   96.34   3514.28
   93.89   3508.31
   95.41   3515.53
   91.61   3433.22
   95.82   3579.68
   90.36   3388.89
   93.64   3513.03
   93.32   3478.04
   ...

$ log from buildkitd
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE stream=11 len=4 incr=32768
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE len=4 (conn) incr=271
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE stream=11 len=4 incr=271
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE len=4 (conn) incr=32496
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE stream=11 len=4 incr=32496
...
```

The registry will update window size of flow control when receives each
frame data. The sender need wait for receiver update the window size if
the sender runs out of buffer of flow control. But the increase value
for buffer by WINDOW_UPDATE frame is too small and slow which impacts
push performance.

Before net/http package provides tunnable value for flow control, we
should disable http2 for https request.

And with this commit, the performance will be better like:

```
$ ifstat -i enp0s3
      enp0s3
 KB/s in  KB/s out
    0.56      0.61
   16.13      5.55
   18.89      9.23
  218.84   7832.80
  338.56  13074.04
  302.39  11713.83
  231.62   8964.60
  356.50  13504.02
  298.14  11401.81
  311.24  11783.26
  333.01  12710.17
  329.64  12630.40
  305.87  11662.04
  292.53  11118.04
```

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2020-03-31 17:59:13 +08:00
Tonis Tiigi 56f76a5621 resolver: support self signed certificates
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-03-16 22:09:27 -07:00
Tõnis Tiigi b9ef26d15f
Merge pull request #1403 from Container-Projects/master
optimization debian package manager tweaks
2020-03-11 17:34:11 -07:00
Pratik Raj 2910de68b1 optimization debian package manager tweaks
By default, Ubuntu or Debian based "apt" or "apt-get" system installs recommended but not suggested packages .

By passing "--no-install-recommends" option, the user lets apt-get know not to consider recommended packages as a dependency to install.

This results in smaller downloads and installation of packages .

Refer to blog at [Ubuntu Blog](https://ubuntu.com/blog/we-reduced-our-docker-images-by-60-with-no-install-recommends) .

Signed-off-by: Pratik Raj <rajpratik71@gmail.com>
2020-03-11 15:27:54 +05:30
Tonis Tiigi 2c3cf11fde resolver: update to new registryhosts based config
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-03-05 19:56:43 -08:00
Akihiro Suda 7b579cdb98 vendor: update containerd (and various packages)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-02-28 10:24:10 +09:00
Tonis Tiigi d1458a6587 update supported platforms without restart
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-02-26 08:15:42 -08:00
Tonis Tiigi a60ecfa4ae vendor: restore dependency versions
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-02-24 17:31:01 -08:00
Zach Badgett 1efe7b145d Fix docker dependencies
Signed-off-by: Zach Badgett <zach.badgett@gmail.com>
2020-02-24 15:47:32 -08:00
Tonis Tiigi 8f52339933 dockerfile: add test for whitelisted devices
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-01-30 18:43:04 -08:00
Tonis Tiigi 572a2b5718 entitlements: mount loop devices relative to next free device
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-01-30 13:47:44 -08:00
Tonis Tiigi fc186a8b89 oci: mount whitelist of devices on insecure security mode
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-01-30 13:07:29 -08:00
Paul "TBBle" Hampson e11b881c12 Set sensible defaults for Windows installations
Non-packaged execution will need this to be overridden anyway, and it
avoids a surprise "Drop state data into the current working directory"
event.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-01-30 00:34:44 +11:00
Tõnis Tiigi 8a267827cd
Merge pull request #1316 from coryb/goroutine-leak
ensure context is cancelled to prevent goroutine leaks from grpc.newClientStream
2020-01-16 17:35:11 -08:00
岁丰 3a0bfbe1ca fix legacy config media type application/octet-stream
Signed-off-by: genglu <genglu.gl@antfin.com>
2020-01-13 11:20:00 +08:00
Cory Bennett 463fc8d1b8 ensure context is cancelled to prevent goroutine leaks from grpc.newClientStream
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-01-07 15:22:59 -08:00
Wei Fu 1845c31ee7 push: update dist repo source for mountFrom
For new built layers, it can be pushed into remote registry. However, it
might not be pulled as base layer. Before this commit, the dist repo
source label for mountFrom call only is updated at pull stage. Without
mountFrom, it will take long time to push existing layers which can be
mountFrom. Based on this case, we should update dist repo source after
pushing layer successfully.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-12-07 00:37:19 +08:00
Akihiro Suda 5938170b84 hack: rename Dockerfiles
Fix https://github.com/moby/buildkit/issues/1208

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-10-18 17:21:48 +09:00
Tonis Tiigi 339d4b2fef leaseutil: mark temporary leases with timestamps
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-10-16 10:35:50 -07:00
Tonis Tiigi 6be136db6e cleanup unused code
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-10-16 10:33:01 -07:00
Tonis Tiigi a9209caa93 clean up temporary leases on startup after crash
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-10-16 10:32:57 -07:00
Tonis Tiigi 688e2c2272 cache: update components to new lease based cache manager
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-10-16 10:32:04 -07:00
Akihiro Suda e2ea141383 hack: remove legacy Dockerfile
Fix #1022

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-10-13 00:10:27 +09:00
Derek McGowan d5108d038d Update to containerd 1.3.0
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-09-30 14:45:18 -07:00
Tonis Tiigi 211e20a587 vendor: update containerd to v1.3.0-rc.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-09-20 14:59:57 -07:00
Sam Whited e90be385b4 all: factor only required methods out of Sandbox
Signed-off-by: Sam Whited <sam@samwhited.com>
2019-08-27 16:54:20 -05:00
Tõnis Tiigi df7e93a159
Merge pull request #1147 from tonistiigi/cross-repo-push
add cross-repo push support
2019-08-27 10:48:43 -07:00
Sam Whited 93687ae5bf util/testutil/integration: s/TestConf/testConf/
Signed-off-by: Sam Whited <sam@samwhited.com>
2019-08-26 07:56:33 -05:00
Tonis Tiigi 481d39c176 add cross-repo push support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-22 17:25:13 -07:00
Tonis Tiigi fdce39046a vendor: update containerd to v1.3.0-beta.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-21 20:22:55 -07:00
Tonis Tiigi e86374886a util: check for localhost for plainhttp
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-20 17:03:04 -07:00
Tonis Tiigi be3635e8cb util: remove unused dockerexporter package
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-20 17:03:04 -07:00
Tonis Tiigi b407790852 update containerd daemon to 1.2.7 and old to 1.1.7
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-20 17:02:43 -07:00
Tonis Tiigi 615347271d vendor: update containerd to 1.3.0-beta.0
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-20 17:00:47 -07:00
Akihiro Suda a545b2dd5b
Merge pull request #1120 from tonistiigi/ci-caching
Use buildkit cache and parallelization in CI
2019-08-21 08:21:11 +09:00
Tonis Tiigi 2cea1107d6 travis: add ci caching
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-17 09:13:06 -07:00
Tonis Tiigi b8284301fd integration: detect double released and leaked mountpoints
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-16 08:03:26 -07:00
Tibor Vass 4c09c48d1d
Merge pull request #1114 from tonistiigi/cni-refactor
network: refactor cni support to multiple packages
2019-08-06 18:56:11 -07:00
Tonis Tiigi 57d19b1208 network: refactor cni support to multiple packages
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-06 12:52:37 -07:00
Tõnis Tiigi 940aab83bc
Merge pull request #1112 from tiborvass/detect-i386
binfmt_misc: add detection for 386
2019-08-06 12:27:03 -07:00
Tibor Vass d8f78dca34 binfmt_misc: add detection for 386
Detects x86 32-bit architectures

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-08-06 01:18:55 +00:00
Tonis Tiigi 8ad218804e correct int64 alignment
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-05 17:20:23 -07:00
Tõnis Tiigi 2439a117e4
Merge pull request #1103 from tonistiigi/power-detect
binfmt_misc: add support for s390x and ppc64le
2019-07-31 10:19:28 -07:00
Tonis Tiigi a805bc9d29 flightcontrol: mitigate retry error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-30 16:05:46 -07:00
Tonis Tiigi 43daf62710 binfmt_misc: add support for s390x and ppc64le
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-30 14:55:46 -07:00
Akihiro Suda d750cb188a
Merge pull request #1073 from tonistiigi/cni-net
add cni networking support
2019-07-19 10:47:27 +09:00
Tonis Tiigi 61a4e20547 util: patch atomic 64bit alignment
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-16 17:23:36 -07:00
Tonis Tiigi acbdfc2851 util: add warning if network fallback is used
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-15 10:53:18 -07:00
Tonis Tiigi 954b73dae1 network: move ns creation from reexec to linkname
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-12 14:57:10 -07:00
Tonis Tiigi e38512c056 testutil: add logs printing on error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-12 11:00:07 -07:00
Tonis Tiigi 653c91e114 network: add non-lazy cni network init
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-12 11:00:02 -07:00
Tonis Tiigi e7759a861b client: add cni network tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-10 18:00:06 -07:00
Tonis Tiigi bc7a6f2556 add cni networking support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-10 14:42:30 -07:00
zabio3 be2691d06c using raw string literal
regular expressions have their own escape sequences, raw strings can improve their readability.
Signed-off-by: Tomohiro Kusumoto <zabio1192@gmail.com>
2019-06-30 12:24:25 +09:00
Tibor Vass 914d86758c
Merge pull request #1024 from tonistiigi/gcr-workaround
pull: add cache for gcr bug
2019-06-17 10:36:49 -07:00
Tonis Tiigi 964d73da58 binfmt_misc: add riscv64 detection
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-06-03 23:21:25 -07:00
Tonis Tiigi 42cd080667 imageutil: keep config and manifest lease for time based cache
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-05-28 19:18:56 -07:00
Tonis Tiigi ab23a1ab1b pull: add cache for gcr bug
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-05-28 18:51:55 -07:00
Tonis Tiigi d72c19071a imageutil: use leases on fetching config
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-05-28 14:39:05 -07:00
Tonis Tiigi 8e25187cc7 pull: fix content blobs deletion on pull race
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-05-28 14:39:05 -07:00
msg 78a8e643f6 Add missing locks around broadcasts
Signed-off-by: Mark Gordon <msg555@gmail.com>
2019-05-10 18:27:37 -04:00
Tonis Tiigi 0dfe132df3 binfmt_misc: detect arm64 properly
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-17 16:14:08 -07:00
Tonis Tiigi c277824a71 progressui: allow vertex reset
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-09 17:43:52 -07:00
Tonis Tiigi a61d7a40a2 binfmt_misc: fix building on windows
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-03 00:23:44 -07:00
Tonis Tiigi e1d2033bf1 progressui: allow unclipped output with env
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-02 16:51:37 -07:00
Tonis Tiigi 770dbc81bf progressui: print error again on plain mode
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-02 16:45:17 -07:00
Tonis Tiigi d451b66bae progressui: improve plain progress
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-02 16:41:07 -07:00
Tonis Tiigi 9b081e607d progressui: fix padding length
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-02 08:50:25 -07:00
Tonis Tiigi 8908d4934a progressui: prioritize already shown logs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-02 08:45:40 -07:00
Tonis Tiigi 9363cbb19a progressui: limit refresh rate and make configurable
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-02 08:19:08 -07:00
Tonis Tiigi 9e73ce6da2 progressui: add logs to the terminal output
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-01 19:48:59 -07:00
Tonis Tiigi 12f178c045 progressui: wrap out of screen incomplete rows
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-01 13:51:50 -07:00
Tonis Tiigi 823e17eccd progressui: cache jobs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-01 10:57:45 -07:00
Tõnis Tiigi 758c61e873
Merge pull request #570 from kunalkushwaha/entirlement-revised
security entitlement support
2019-03-28 16:14:06 -07:00
Tonis Tiigi 01a8de7d74 integration: secmode switch support and security testcase
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Kunal Kushwaha <kunal.kushwaha@gmail.com>
2019-03-27 13:57:03 +09:00
Kunal Kushwaha a2bbb5ff39 security entitlement support
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2019-03-27 13:57:03 +09:00
Tibor Vass fe0b8a4ec7
Merge pull request #902 from tonistiigi/itegration-update
integration: generalize pkg
2019-03-26 16:35:12 -07:00
Tõnis Tiigi 2453e6f388
Merge pull request #854 from chendave/hint
Check the validity of the platforms
2019-03-24 09:09:08 -07:00
Dave Chen aa31ff073f Check the validity of the platforms
platforms can still be added but some warning message
will be emitted if the platform cannot pass the validity
check.

Signed-off-by: Dave Chen <dave.chen@arm.com>
2019-03-23 21:09:42 -07:00
Tonis Tiigi 7e52b33bf8 integration: generalize pkg
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-21 23:49:08 -07:00
Tibor Vass 9a9e755588
Merge pull request #887 from tonistiigi/push-by-digest
exporter: allow push by digest in image exporter
2019-03-20 19:13:09 -07:00
Tibor Vass 9e8dfa72d3
Merge pull request #885 from tonistiigi/content-gc
content: fix setting wrong gc labels on config query
2019-03-20 18:52:41 -07:00
Akihiro Suda db01ff5df5
Merge pull request #891 from chendave/deprecated
Stop using the deprecated way for build
2019-03-20 13:46:02 +09:00
Dave Chen c7ac2f392a Stop using the deprecated way for build
Signed-off-by: Dave Chen <dave.chen@arm.com>
2019-03-19 19:01:31 -07:00
hpandeycodeit 271f59fe60 Added support for mutiple tags - docker build
Signed-off-by: hpandeycodeit <hpandey@pivotal.io>
2019-03-18 17:23:18 -07:00
Tonis Tiigi 37bfc6546c exporter: allow push by digest in image exporter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-18 16:00:48 -07:00
Tonis Tiigi 2dc7a133a2 content: fix setting wrong gc labels on config query
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-18 12:00:09 -07:00
Tõnis Tiigi 0c747f4276
Merge pull request #861 from tonistiigi/disable-gc
integration: disable gc
2019-03-17 18:01:29 -07:00
Akihiro Suda cd89a8ce3d go 1.12
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-03-17 04:38:23 +09:00
zabio3 f02cc76b2a Fix typo
Signed-off-by: Tomohiro Kusumoto <zabio1192@gmail.com>
2019-03-16 17:39:02 +09:00
Dave Chen 156754dc78 Fix the warning for the file not end with newline
arm-linux-gnueabihf-as -o exit.o exit.arm.s

exit.arm.s: Assembler messages:
exit.arm.s: Warning: end of file not at end of a line; newline inserted

Signed-off-by: Dave Chen <dave.chen@arm.com>
2019-03-12 01:49:38 -07:00
Tonis Tiigi aece2ac1aa integration: disable gc
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-10 23:45:48 -07:00
Tonis Tiigi 632797cef0 vendor: udpate fsutil to b4281fa
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-05 17:34:01 -08:00
Akihiro Suda 2513fcf7d8
Merge pull request #840 from tonistiigi/qemu-detection
util: add binfmt_misc binary detection
2019-03-02 15:38:08 +09:00
Tonis Tiigi fb73a926c0 util: add binfmt_misc binary detection
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-02-28 11:00:04 -08:00
Derek McGowan edd603e4cd Update containerd vendor
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-02-13 11:03:48 -08:00
Tonis Tiigi 97eff70c5e inline remote cache support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-01-21 19:39:15 -08:00
Tonis Tiigi 88c81ee6d5 testutil: use different containerd debug sock
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-16 19:46:31 -07:00
Akihiro Suda 0d80bd17a3
Merge pull request #687 from tonistiigi/dockerfile-testing
dockerfile: add testing external dockerfile features
2018-10-16 15:58:55 +09:00
Tonis Tiigi b15990f799 hack: add better caching
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-15 22:27:13 -07:00
Tonis Tiigi 1be3e43527 testutil: add testing a random worker
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-15 22:27:13 -07:00
Tonis Tiigi 706fa2556a dockerfile: add testing external dockerfile features
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-15 22:27:07 -07:00
Akihiro Suda 048130d1d0 simplify rootless
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-10-16 14:05:58 +09:00
Tonis Tiigi 1f51f5ea11 testutil: avoid excessive daemons in integration tests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-11 16:47:35 -07:00
Tonis Tiigi 43bd294446 integration: add setsid for sigterm
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-10 23:26:36 -07:00
Tonis Tiigi 89e8097d4c fix setting uncompressed label on content
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-08 23:11:25 -07:00
Tonis Tiigi 58a6203857 contentutil: add ref lock on push
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-05 17:21:38 -07:00
Tonis Tiigi 8360d73946 testutil: avoid overriding cleanup to nil on error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-05 17:21:34 -07:00
Tibor Vass a0c740485c
Merge pull request #663 from tonistiigi/opentracing-fix
solver: fix opentracing providers
2018-10-03 11:39:29 -07:00
Tibor Vass ae84fa2c9a flightcontrol: add exponential backoff to Group.Do
This patch fixes a stack overflow panic by calling Do in a loop
instead of relying on recursion, with an added exponential
backoff which errors out if greater than 3 seconds.

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-10-03 17:56:18 +00:00
Tonis Tiigi b26e666db9 solver: fix opentracing providers
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-10-02 23:11:17 -07:00
Tõnis Tiigi 39404586a5
Merge pull request #622 from tonistiigi/df-modes-switch
dockerfile: enable frontend selection in integration tests
2018-09-20 14:44:11 -07:00
Tibor Vass 9890dda814
Merge pull request #634 from tonistiigi/schema1-config-fix
imageutil: fix getting schema1 configs
2018-09-19 15:50:52 -07:00
Tonis Tiigi fe71cd39bd imageutil: fix getting schema1 configs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-19 14:09:32 -07:00
Tibor Vass cb0d51cca3
Merge pull request #630 from tonistiigi/cleanup
golint and misspell cleanup
2018-09-19 12:01:45 -07:00
Tonis Tiigi a75983ddb9 integration: pass mirrored images from test
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-18 22:07:38 -07:00
Tonis Tiigi a0d7ed9da0 integration: add passing matrices of options
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-18 22:07:38 -07:00
Tonis Tiigi 0940cdc6fe update golint comments
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-18 22:06:47 -07:00
Tonis Tiigi 301da72ce3 misspell fixes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-18 22:06:38 -07:00
Tonis Tiigi 087e11eedd integration: fix config permissions for rootless
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-11 21:53:40 -07:00
Tonis Tiigi fe8f2c9910 integration: fix cleanup of registry data
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-11 18:37:27 -07:00
Tonis Tiigi 75ea12b6c9 integration: use local registry mirrors
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-11 18:37:21 -07:00
Tonis Tiigi ecd4a22546 resolver: add config support for mirrors/plainhttp
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-07 18:44:45 -07:00
Akihiro Suda 756ca6cc6d
Merge pull request #602 from tonistiigi/gc
automatic GC
2018-09-05 10:47:21 +09:00
Ian Campbell 7cb70e1c31 Initialise workers' label maps before assigning.
Otherwise:

    panic: assignment to entry in nil map

    goroutine 1 [running]:
    main.applyOCIFlags(0xc4200e71e0, 0xc420400000, 0x0, 0x0)
    	/go/src/github.com/moby/buildkit/cmd/buildkitd/main_oci_worker.go:97 +0x1ac
    main.ociWorkerInitializer(0xc4200e71e0, 0xc4204104e0, 0xc420400000, 0x43409b, 0x12, 0xc42026b0f8, 0x4337fc, 0xc420000180)
    	/go/src/github.com/moby/buildkit/cmd/buildkitd/main_oci_worker.go:118 +0x50
    main.newWorkerController(0xc4200e71e0, 0xc4204104e0, 0xc420400000, 0xc420422000, 0xe5dc54, 0x11)
    	/go/src/github.com/moby/buildkit/cmd/buildkitd/main.go:520 +0x324
    main.newController(0xc4200e71e0, 0xc420400000, 0x1c0, 0x0, 0x0)
    	/go/src/github.com/moby/buildkit/cmd/buildkitd/main.go:489 +0xdc
    main.main.func3(0xc4200e71e0, 0x0, 0x0)
    	/go/src/github.com/moby/buildkit/cmd/buildkitd/main.go:203 +0x3dd
    github.com/moby/buildkit/vendor/github.com/urfave/cli.HandleAction(0xcdd420, 0xe93e98, 0xc4200e71e0, 0xc4200e71e0, 0xc42026b888)
    	/go/src/github.com/moby/buildkit/vendor/github.com/urfave/cli/app.go:502 +0xc8
    github.com/moby/buildkit/vendor/github.com/urfave/cli.(*App).Run(0xc4201b6540, 0xc4200300a0, 0xa, 0xa, 0x0, 0x0)
    	/go/src/github.com/moby/buildkit/vendor/github.com/urfave/cli/app.go:268 +0x60c
    main.main()
    	/go/src/github.com/moby/buildkit/cmd/buildkitd/main.go:238 +0xc64

Also add some random labels to the integration sandbox (which I have confirmed
is enough to trigger this issue before the fix).

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-09-03 11:35:36 +01:00
Tonis Tiigi 8fea3f25e1 util: add a nonblocking throttle variant
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-31 16:36:58 -07:00
Tonis Tiigi c524709f4a buildkitd: add config file support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-30 16:11:55 -07:00
Derek McGowan 279a144686 Update containerd vendor
Picks up platform changes

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-28 14:26:14 -07:00
Akihiro Suda 557bba11c9 containerd: disable CRI plugin
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-08-23 18:14:57 +09:00
Akihiro Suda b90615d34d integration: print containerd logs on failure
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-08-23 18:14:57 +09:00
Tibor Vass 2f0c048493 network: move handling of NetMode_UNSET to Default()
This allows other workers to implement their own behavior for NetMode_UNSET

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-08-23 00:06:06 +00:00
Michael Crosby b97bc71adb Refactor networking with ns paths
This fixes the issues where buildkit and callers do not have to be a
subpreaper in order to use networking.  I can add CNI provider later,
with a hidden sub command to create a new network namespace and bind
mount it to buildkit's state dir.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-21 13:37:47 -04:00
Tonis Tiigi 130f5f5ab0 solver: net host with basic entitlements support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-09 14:03:35 -07:00
Tõnis Tiigi 785436a312
Merge pull request #563 from ijc/progressui-custom-operation
progressui: allow caller to customise "Building" string
2018-08-09 10:49:29 -07:00
Ian Campbell ccc559bc43 progressui: allow caller to customise "Building" string
In clients which are doinging multiple builds or phases it can be useful to say
something more specific here (e.g. "Building first image", "Probing" etc)

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-08 16:29:01 +01:00
Kunal Kushwaha 765f1b64b9 executor: allow network providers
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-07 11:51:56 -07:00
Tonis Tiigi e971b7cddf pull: handle resolvemode
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 18:20:55 -07:00
Tonis Tiigi f719aadcba dockerfile: add tests for multi-platform exports
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 16:01:12 -07:00
Tonis Tiigi 6be1257f5d export: manifest list support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-25 16:01:12 -07:00
Tõnis Tiigi 5098434f33
Merge pull request #530 from tonistiigi/winlayers-panic
winlayers: discard reader on apply errors
2018-07-25 12:59:35 -07:00
Ian Campbell e0196c9a12 apicaps: record the correct caps when constructing a CapSet
We need to capture `c` within the loop, otherwise all of the entries of `m` end
up pointing to the same thing, which is the last cap in `range caps`.

The symptom was that all caps were enabled/disabled based on the last cap's
setting and not their own.

Add a test for this.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-07-25 14:29:16 +01:00
Tonis Tiigi 384263c68d winlayers: discard reader on apply errors
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-20 18:37:26 -07:00
Akihiro Suda a9dcf63f45 integration: skip rootless if unsupported
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-07-20 19:18:51 +09:00
Tonis Tiigi bc765861be diff: implement windows layer support for linux
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-16 16:33:21 -07:00
Tõnis Tiigi aee2723382
Merge pull request #480 from ijc/less-chatty-non-console-progress
Reduce verbosity of non-console solve status.
2018-07-10 08:25:12 -07:00
Ian Campbell dbd3d86a35 Reduce verbosity of non-console solve status.
Only report progress at most every 5s or 5% completion, which ever comes first.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-07-10 16:06:31 +01:00
Tonis Tiigi 90d164ced5 gateway: inital caps detection support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-09 16:32:38 -07:00
Tonis Tiigi db5b0f1b9a api: add caps support for versioning
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-09 16:32:34 -07:00
Tõnis Tiigi 77e4fcd0ec
Merge pull request #491 from frezbo/feature/fix-datatype-mismatch
Fixing mismatch
2018-07-06 11:53:14 -07:00
Noel Georgi 8753e76a4d
Fixing mismatch
Signed-off-by: Noel Georgi <18496730+frezbo@users.noreply.github.com>
2018-07-06 17:48:17 +05:30
Akihiro Suda 864fa7465e integration: rename "/opt/containerd-1.0/bin/containerd" worker to "containerd-1.0"
Having '/' in a worker name is confusing.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-07-06 15:32:02 +09:00
Tõnis Tiigi cda0a64eca
Merge pull request #486 from AkihiroSuda/update-rootless
rootless: refactor libcontainer_specconv & add integration tests
2018-07-05 11:26:26 -07:00
Akihiro Suda 580dbe08b9 rootless: add integration tests
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-07-04 19:27:54 +09:00
Akihiro Suda 72327724c0 rootless: refactor libcontainer_specconv
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-07-04 19:27:54 +09:00
Akihiro Suda 80d2f820f9 decopule cache/remotecache from registry
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-07-04 17:38:09 +09:00
Tõnis Tiigi bbcd8adce0
Merge pull request #484 from tonistiigi/status-dupe-fix
pull: do not send duplicate status for completed jobs
2018-07-03 15:27:22 -07:00
Tonis Tiigi 07d8189603 progressui: fix log line separation bug
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-03 14:43:26 -07:00
Tonis Tiigi e1471c5682 pull: do not send duplicate status for completed jobs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-07-03 12:14:09 -07:00
Anda Xu f00c1bc2cb re-enable debug messages
Signed-off-by: Anda Xu <anda.xu@docker.com>
2018-07-02 11:05:56 -07:00
Tõnis Tiigi f3b743bd13
Merge pull request #467 from dmcgowan/export-oci-media-types
Support creating and pushing OCI types
2018-06-28 14:46:34 -07:00
Derek McGowan 618f34a1f9 Support creating and pushing OCI types
Adds image and oci exporter option "oci-mediatypes"
Ensures that the images created in the content store
have the correct type which matches the manifest.
Sets the correct media type on the descriptor in push from
reading the type specified in the manifest.
Removes use of distribution manifest packages.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-06-28 11:23:35 -07:00
Tonis Tiigi f9383c48a3 source: image source platform support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-25 10:39:17 -07:00
Akihiro Suda 7690ff1d34 vendor runc ad0f5255060d36872be04de22f8731f38ef2d7b1
For https://github.com/opencontainers/runc/pull/1819

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-06-16 03:45:49 +09:00
Tõnis Tiigi 7943598281
Merge pull request #439 from AkihiroSuda/vendor-containerd-20180606
update containerd library
2018-06-08 10:33:10 -07:00
Akihiro Suda 47a00b84fe update containerd library
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-06-08 21:44:17 +09:00
Tonis Tiigi 2037b9540b progressui: constant prefix index for grep
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-06 14:31:48 -07:00
Tonis Tiigi 39e19516fc progressui: add better streaming text build status
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-06 14:31:42 -07:00
Jess Frazelle 48200d7b5e
support when build without seccomp build tag
Signed-off-by: Jess Frazelle <acidburn@microsoft.com>
2018-06-05 14:27:32 -04:00
Akihiro Suda adef0dedef oci-worker: experimental support for rootless mode
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-05-31 16:05:13 +09:00
Akihiro Suda 8b131af00d
Merge pull request #415 from tonistiigi/window-size
progressui: handle zero window size
2018-05-26 06:59:52 +09:00
Tonis Tiigi 5727898274 progressui: handle zero window size
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-25 14:38:06 -07:00