Commit Graph

28 Commits (201ba2b0e35799cfbd5ba043d187e499549613ea)

Author SHA1 Message Date
Tonis Tiigi aa29e77294 fix import cycle in integration test pulls
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-11 21:21:08 -08:00
Tonis Tiigi ffe71a1ea9 add retry handlers to push/pull
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-11 20:39:01 -08:00
Tõnis Tiigi be424a99ea
Merge pull request #1728 from earthly/vlad/authorizer-errors
Prevent context canceled errors from being permanent in authorizer
2020-10-15 17:46:14 -07:00
Vlad A. Ionescu d127edf990 Prevent context canceled errors from being permanent in authorizer
Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>

Check if context is canceled before returning cached token or err.

Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>

Fix possible race conditions

Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>

Fix block

Signed-off-by: Vlad A. Ionescu <vladaionescu@users.noreply.github.com>
2020-10-15 15:11:59 -07:00
Tonis Tiigi 1f94445456 auth: fetch tokens from client side
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-09-15 13:40:30 -07:00
Ilya Dmitrichenko 6e8fde8431
vendor: update containerd to d4e7820
Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
2020-09-07 10:40:09 +01:00
Tonis Tiigi d33756e17b vendor: update containerd to efa0e809
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-30 23:04:25 -07:00
Tonis Tiigi e6500927d2 resolver: handle nil group properly
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-13 17:27:25 -07:00
Tonis Tiigi e322304c07 resolver: avoid retrying fatal error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-13 17:27:25 -07:00
Tonis Tiigi 8f8dccf837 resolver: add docs to new functions
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-13 17:27:25 -07:00
Tonis Tiigi 5f69909037 resolver: switch post/get order on requesting token
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-13 17:27:25 -07:00
Tonis Tiigi b474dbf55f resolver: clean up unused resolver pool
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-13 17:27:18 -07:00
Tonis Tiigi 21c4ab8621 resolver: fix token expiring issues
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-13 17:27:18 -07:00
Tonis Tiigi f1cd79bf65 resolver: add better pooling and custom authenticator
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-13 17:27:11 -07:00
Tonis Tiigi 22061b1ec9 resolver: reenable keep-alive
Was disabled with http2 but shouldn’t have been.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-02 09:32:46 -07:00
Tonis Tiigi 5e08a24e88 resolver: make sure authorizer is not regenerated
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-02 09:32:40 -07:00
Tonis Tiigi cdf424f677 resolver: enable tracing on default transport
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-29 23:19:52 -07:00
Tonis Tiigi 67044875c4 resolver: allow setting both insecure tls and http
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-26 18:29:45 -07:00
Tonis Tiigi a8c2137598 resolver: add credentials cache
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-05 12:07:21 -07:00
Tonis Tiigi 05013a663b pull: fix session updating on resolver
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-05 00:15:33 -07:00
Tonis Tiigi 214aa5dbcf pull: allow separate sessions for different parts of pull
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-05 00:15:33 -07:00
Tonis Tiigi 2e9987ad16 session: track sessions with a group construct
Avoid hidden session passing and allow one session to drop when
multiple builds share a vertex.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-05 00:15:25 -07:00
Tonis Tiigi 37b8832d00 upgrade errors checks to Is()
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-18 22:53:38 -07:00
Wei Fu 9f063f6b61 resolver: disable http2 for pushing
The golang net/http package uses http2 client to serve https by default,
if let Transport.TLSNextProto is nil. And net/http package doesn't
provide tunnable value for http2 flow control which will limit push
performance.

Before this commit, use GODEBUG="http2debug=1" buildkitd to pushing
one image from dockerfile like

```
$ about 700MB
FROM scratch
ADD ./golang-1.13.0-stretch.tar.gzip /
```

and use ifstat to monitor network interface and found that

```
$ ifstat -i enp0s3
      enp0s3
 KB/s in  KB/s out
    0.47      0.67
    0.44      0.51
   19.72     11.56
   62.25   2184.41
   96.34   3514.28
   93.89   3508.31
   95.41   3515.53
   91.61   3433.22
   95.82   3579.68
   90.36   3388.89
   93.64   3513.03
   93.32   3478.04
   ...

$ log from buildkitd
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE stream=11 len=4 incr=32768
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE len=4 (conn) incr=271
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE stream=11 len=4 incr=271
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE len=4 (conn) incr=32496
2020/03/31 17:40:33 http2: Transport received WINDOW_UPDATE stream=11 len=4 incr=32496
...
```

The registry will update window size of flow control when receives each
frame data. The sender need wait for receiver update the window size if
the sender runs out of buffer of flow control. But the increase value
for buffer by WINDOW_UPDATE frame is too small and slow which impacts
push performance.

Before net/http package provides tunnable value for flow control, we
should disable http2 for https request.

And with this commit, the performance will be better like:

```
$ ifstat -i enp0s3
      enp0s3
 KB/s in  KB/s out
    0.56      0.61
   16.13      5.55
   18.89      9.23
  218.84   7832.80
  338.56  13074.04
  302.39  11713.83
  231.62   8964.60
  356.50  13504.02
  298.14  11401.81
  311.24  11783.26
  333.01  12710.17
  329.64  12630.40
  305.87  11662.04
  292.53  11118.04
```

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2020-03-31 17:59:13 +08:00
Tonis Tiigi 56f76a5621 resolver: support self signed certificates
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-03-16 22:09:27 -07:00
Tonis Tiigi 2c3cf11fde resolver: update to new registryhosts based config
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-03-05 19:56:43 -08:00
Tonis Tiigi e86374886a util: check for localhost for plainhttp
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-20 17:03:04 -07:00
Tonis Tiigi ecd4a22546 resolver: add config support for mirrors/plainhttp
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-07 18:44:45 -07:00