Currently, `docker build --progress=plain` truncates output, which makes it
difficult to debug problems during build (the step that failed may not be the
cause of the failure).
For example, in the following build, the output of the `RUN echo ...` is truncated:
DOCKER_BUILDKIT=1 docker build --no-cache --progress=plain -t foo -<<EOF
FROM busybox
RUN echo "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas ut lorem nec leo euismod vestibulum. Donec tortor nisl, iaculis at vehicula vel, interdum eu orci. Integer velit lacus, congue id magna eu, mollis accumsan augue. Aliquam non venenatis risus, eu posuere libero. Vestibulum ante ipsum primis in faucibus orci luctus." > /somewhere
RUN echo "something went wrong"; exit 1
EOF
#5 [2/3] RUN echo "Lorem ipsum dolor sit amet, consectetur adipiscing elit....
#5 DONE 0.2s
#6 [3/3] RUN echo "something went wrong"; exit 1
#6 0.211 something went wrong
#6 ERROR: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully
------
> [3/3] RUN echo "something went wrong"; exit 1:
------
failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully
While there is an existing `PROGRESS_NO_TRUNC` environment variable, I think that
this should be the default if the user opted to use `--progress=plain` (or in
situations where no TTY is attached, which could be in CI).
This patch changes the default to disable truncating in those situations, but
allowing users to opt-out by setting `PROGRESS_NO_TRUNC=0`
With this change the same build looks like this:
#5 [2/3] RUN echo "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas ut lorem nec leo euismod vestibulum. Donec tortor nisl, iaculis at vehicula vel, interdum eu orci. Integer velit lacus, congue id magna eu, mollis accumsan augue. Aliquam non venenatis risus, eu posuere libero. Vestibulum ante ipsum primis in faucibus orci luctus." > /somewhere
#5 sha256:b2f0f47d63911ee55e7cf17c81007310e28190b5be84aa1a7869ba90786d5cee
#5 DONE 0.2s
#6 [3/3] RUN echo "something went wrong"; exit 1
#6 sha256:c037b34bb998ae7d30572b489286da14df87e1478adf6d0c5c71c79b84b11bcc
#6 0.293 something went wrong
#6 ERROR: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully
------
> [3/3] RUN echo "something went wrong"; exit 1:
------
failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c echo "something went wrong"; exit 1]: runc did not terminate sucessfully
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The golang net/http package uses http2 client to serve https by default,
if let Transport.TLSNextProto is nil. And net/http package doesn't
provide tunnable value for http2 flow control which will limit push
performance.
Before this commit, use GODEBUG="http2debug=1" buildkitd to pushing
one image from dockerfile like
```
$ about 700MB
FROM scratch
ADD ./golang-1.13.0-stretch.tar.gzip /
```
and use ifstat to monitor network interface and found that
```
$ ifstat -i enp0s3
enp0s3
KB/s in KB/s out
0.47 0.67
0.44 0.51
19.72 11.56
62.25 2184.41
96.34 3514.28
93.89 3508.31
95.41 3515.53
91.61 3433.22
95.82 3579.68
90.36 3388.89
93.64 3513.03
93.32 3478.04
...
$ log from buildkitd
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE stream=11 len=4 incr=32768
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE len=4 (conn) incr=271
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE stream=11 len=4 incr=271
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE len=4 (conn) incr=32496
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE stream=11 len=4 incr=32496
...
```
The registry will update window size of flow control when receives each
frame data. The sender need wait for receiver update the window size if
the sender runs out of buffer of flow control. But the increase value
for buffer by WINDOW_UPDATE frame is too small and slow which impacts
push performance.
Before net/http package provides tunnable value for flow control, we
should disable http2 for https request.
And with this commit, the performance will be better like:
```
$ ifstat -i enp0s3
enp0s3
KB/s in KB/s out
0.56 0.61
16.13 5.55
18.89 9.23
218.84 7832.80
338.56 13074.04
302.39 11713.83
231.62 8964.60
356.50 13504.02
298.14 11401.81
311.24 11783.26
333.01 12710.17
329.64 12630.40
305.87 11662.04
292.53 11118.04
```
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Tonis Tiigi
Vlad A. Ionescu
Tibor Vass
Akihiro Suda
Edgar Lee
Sebastiaan van Stijn
Sam Whited
Wei Fu
Xiaofan Zhang
Vlad A. Ionescu
Andrea Luzzardi
Darren Shepherd