The previous implementation had many issues. Eg. on fetch, even if
the data already existed and no remote connections were needed
the request would still be waiting in the queue. Or if two fetches
of same blob happened together they would take up two places in queue
although there was only one remote request.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This is a safer alternative until we figure out why
http.Transport based limiting fails.
Some connections like cache export/import do not have a
domain key atm and these connections use global pool.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Some registries can be flaky and return intermittent 5xx errors. This
change allows those errors to be retried, similarly to network-level
errors.
Note that this needs the upstream containerd fix
https://github.com/containerd/containerd/pull/5276 to work reliably.
This was tested with a registry that was modified to return 504 on every
other manifest PUT. Without the change, exports to the registry fail
every other attempt. With the change and the related containerd change,
exports to the registry always succeed.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
Koichi Shiraishi
Tonis Tiigi
Aaron Lehmann
Derek McGowan
Akihiro Suda