`go test` complains about this tree because the package ends up with no
symbols on non-Linux platforms, e.g., Windows, even though this API is
never be called on non-Linux platforms, according to the compiler.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Although we don't run any integration tests on Windows at this time,
this does mean that the SKIP_INTEGRATION_TESTS env-var works, so we can
run a wider range of non-integration tests without spurious failures.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
There's no Loopback CNI plugin for Windows as far as I can see, so I'm
assuming it's not needed.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
The `WithLoNetwork` and `WithConfFile` options consume the
`WithPluginDir` and `WithInterfacePrefix` settings, so they must come
first.
This fixes the issue that the provided plugin directory was not used for
the provided configuration file. This would have been more visible, but
the default `InterfacePrefix` is already "eth".
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
The Windows API is completely different, so the common code cannot
create the unique identifier, but must accept what the OS-specific code
does.
For Linux, it's a pathname. In Windows, it's a GUID string.
This pulls in direct usage of Microsoft/hcsshim, which previously was
only an indirect usage via containerd calls from, e.g., localMounter on
Windows.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
A recent change updated the containerd shim built for integ tests to the newer
v2, but the integ test code itself was still hardcoded to look for the older
shim binary and, when that binary wasn't found, skip running the test. This
caused all containerd tests to be skipped.
The fix here updates the binary name to v2 and also removes the branch that
will skip a test when the requirements check fails.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
Before this, the check was always returning that containerd wasn't running and
thus skipping the rest of several test cases.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
The implementation in libcontainer/system is quite complicated,
and we only use it to detect if user-namespaces are enabled.
In addition, the implementation in containerd uses a sync.Once,
so that detection (and reading/parsing `/proc/self/uid_map`) is
only performed once.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
```
[5/5] RUN --mount=target=/go/src/github.com/moby/buildkit gometalinter ...
0.435 util/rootless/specconv/specconv_linux.go:1:⚠️ file is not goimported (goimports)
1.320 cache/manager.go:1:⚠️ file is not goimported (goimports)
1.335 cache/manager_test.go:1:⚠️ file is not goimported (goimports)
1.337 cache/migrate_v2.go:1:⚠️ file is not goimported (goimports)
1.342 cache/refs.go:1:⚠️ file is not goimported (goimports)
1.454 cache/remotecache/registry/registry.go:1:⚠️ file is not goimported (goimports)
2.285 cmd/buildctl/build.go:1:⚠️ file is not goimported (goimports)
3.082 executor/oci/user.go:1:⚠️ file is not goimported (goimports)
4.333 session/content/content_test.go:1:⚠️ file is not goimported (goimports)
4.614 snapshot/containerd/content.go:1:⚠️ file is not goimported (goimports)
4.721 solver/errdefs/vertex.go:1:⚠️ file is not goimported (goimports)
6.066 util/network/cniprovider/cni.go:1:⚠️ file is not goimported (goimports)
ERROR: executor failed running [/bin/sh -c gometalinter --config=gometalinter.json ./...]: buildkit-runc did not terminate successfully
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Currently, `docker build --progress=plain` truncates output, which makes it
difficult to debug problems during build (the step that failed may not be the
cause of the failure).
For example, in the following build, the output of the `RUN echo ...` is truncated:
DOCKER_BUILDKIT=1 docker build --no-cache --progress=plain -t foo -<<EOF
FROM busybox
RUN echo "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas ut lorem nec leo euismod vestibulum. Donec tortor nisl, iaculis at vehicula vel, interdum eu orci. Integer velit lacus, congue id magna eu, mollis accumsan augue. Aliquam non venenatis risus, eu posuere libero. Vestibulum ante ipsum primis in faucibus orci luctus." > /somewhere
RUN echo "something went wrong"; exit 1
EOF
#5 [2/3] RUN echo "Lorem ipsum dolor sit amet, consectetur adipiscing elit....
#5 DONE 0.2s
#6 [3/3] RUN echo "something went wrong"; exit 1
#6 0.211 something went wrong
#6 ERROR: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully
------
> [3/3] RUN echo "something went wrong"; exit 1:
------
failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully
While there is an existing `PROGRESS_NO_TRUNC` environment variable, I think that
this should be the default if the user opted to use `--progress=plain` (or in
situations where no TTY is attached, which could be in CI).
This patch changes the default to disable truncating in those situations, but
allowing users to opt-out by setting `PROGRESS_NO_TRUNC=0`
With this change the same build looks like this:
#5 [2/3] RUN echo "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas ut lorem nec leo euismod vestibulum. Donec tortor nisl, iaculis at vehicula vel, interdum eu orci. Integer velit lacus, congue id magna eu, mollis accumsan augue. Aliquam non venenatis risus, eu posuere libero. Vestibulum ante ipsum primis in faucibus orci luctus." > /somewhere
#5 sha256:b2f0f47d63911ee55e7cf17c81007310e28190b5be84aa1a7869ba90786d5cee
#5 DONE 0.2s
#6 [3/3] RUN echo "something went wrong"; exit 1
#6 sha256:c037b34bb998ae7d30572b489286da14df87e1478adf6d0c5c71c79b84b11bcc
#6 0.293 something went wrong
#6 ERROR: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully
------
> [3/3] RUN echo "something went wrong"; exit 1:
------
failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The golang net/http package uses http2 client to serve https by default,
if let Transport.TLSNextProto is nil. And net/http package doesn't
provide tunnable value for http2 flow control which will limit push
performance.
Before this commit, use GODEBUG="http2debug=1" buildkitd to pushing
one image from dockerfile like
```
$ about 700MB
FROM scratch
ADD ./golang-1.13.0-stretch.tar.gzip /
```
and use ifstat to monitor network interface and found that
```
$ ifstat -i enp0s3
enp0s3
KB/s in KB/s out
0.47 0.67
0.44 0.51
19.72 11.56
62.25 2184.41
96.34 3514.28
93.89 3508.31
95.41 3515.53
91.61 3433.22
95.82 3579.68
90.36 3388.89
93.64 3513.03
93.32 3478.04
...
$ log from buildkitd
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE stream=11 len=4 incr=32768
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE len=4 (conn) incr=271
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE stream=11 len=4 incr=271
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE len=4 (conn) incr=32496
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE stream=11 len=4 incr=32496
...
```
The registry will update window size of flow control when receives each
frame data. The sender need wait for receiver update the window size if
the sender runs out of buffer of flow control. But the increase value
for buffer by WINDOW_UPDATE frame is too small and slow which impacts
push performance.
Before net/http package provides tunnable value for flow control, we
should disable http2 for https request.
And with this commit, the performance will be better like:
```
$ ifstat -i enp0s3
enp0s3
KB/s in KB/s out
0.56 0.61
16.13 5.55
18.89 9.23
218.84 7832.80
338.56 13074.04
302.39 11713.83
231.62 8964.60
356.50 13504.02
298.14 11401.81
311.24 11783.26
333.01 12710.17
329.64 12630.40
305.87 11662.04
292.53 11118.04
```
Signed-off-by: Wei Fu <fuweid89@gmail.com>
By default, Ubuntu or Debian based "apt" or "apt-get" system installs recommended but not suggested packages .
By passing "--no-install-recommends" option, the user lets apt-get know not to consider recommended packages as a dependency to install.
This results in smaller downloads and installation of packages .
Refer to blog at [Ubuntu Blog](https://ubuntu.com/blog/we-reduced-our-docker-images-by-60-with-no-install-recommends) .
Signed-off-by: Pratik Raj <rajpratik71@gmail.com>
Non-packaged execution will need this to be overridden anyway, and it
avoids a surprise "Drop state data into the current working directory"
event.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
For new built layers, it can be pushed into remote registry. However, it
might not be pulled as base layer. Before this commit, the dist repo
source label for mountFrom call only is updated at pull stage. Without
mountFrom, it will take long time to push existing layers which can be
mountFrom. Based on this case, we should update dist repo source after
pushing layer successfully.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Tonis Tiigi
Akihiro Suda
Paul "TBBle" Hampson
Erik Sipsma
Sebastiaan van Stijn
Tibor Vass
Simon Ferquel
Sam Whited
Wei Fu
Pratik Raj
Zach Badgett
岁丰
Cory Bennett
Derek McGowan
Tibor Vass