Commit Graph

35 Commits (d5b7ce35d8edb3f609a097adec070ddb69be65df)

Author SHA1 Message Date
Tonis Tiigi b449431019 imageutil: make mediatype detection more stricter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-17 15:44:18 -08:00
Koichi Shiraishi 671ae38fb7
all: unify the specs-go package import alias to ocispecs
ocispecs means "O"pen "C"ontainer "I"nitiative image-spec/"specs"-go/v1
                      opencontainers          /image-spec/specs-go/v1

Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
2021-08-11 08:29:09 +09:00
Tonis Tiigi f269d00f28 new implementation for limiting tcp connections
The previous implementation had many issues. Eg. on fetch, even if
the data already existed and no remote connections were needed
the request would still be waiting in the queue. Or if two fetches
of same blob happened together they would take up two places in queue
although there was only one remote request.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-07-14 20:41:02 -07:00
Tonis Tiigi a558ac4cce add per domain semaphore to limit concurrent connections
This is a safer alternative until we figure out why
http.Transport based limiting fails.

Some connections like cache export/import do not have a
domain key atm and these connections use global pool.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-07-13 14:08:03 -07:00
Aaron Lehmann d3b96f4e31 Add retry on image push 5xx errors
Some registries can be flaky and return intermittent 5xx errors. This
change allows those errors to be retried, similarly to network-level
errors.

Note that this needs the upstream containerd fix
https://github.com/containerd/containerd/pull/5276 to work reliably.

This was tested with a registry that was modified to return 504 on every
other manifest PUT. Without the change, exports to the registry fail
every other attempt.  With the change and the related containerd change,
exports to the registry always succeed.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-03-26 18:13:45 -07:00
Tonis Tiigi aa29e77294 fix import cycle in integration test pulls
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-11 21:21:08 -08:00
Tonis Tiigi ffe71a1ea9 add retry handlers to push/pull
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-11 20:39:01 -08:00
Erik Sipsma 55cbd19dec Add support for lazily-pulled blobs in cache manager.
This allows the layers of images to only be pulled if/once they are actually
required.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-08-05 17:18:43 -07:00
岁丰 3a0bfbe1ca fix legacy config media type application/octet-stream
Signed-off-by: genglu <genglu.gl@antfin.com>
2020-01-13 11:20:00 +08:00
Tonis Tiigi 339d4b2fef leaseutil: mark temporary leases with timestamps
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-10-16 10:35:50 -07:00
Tonis Tiigi a9209caa93 clean up temporary leases on startup after crash
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-10-16 10:32:57 -07:00
Tonis Tiigi 688e2c2272 cache: update components to new lease based cache manager
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-10-16 10:32:04 -07:00
Tonis Tiigi 42cd080667 imageutil: keep config and manifest lease for time based cache
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-05-28 19:18:56 -07:00
Tonis Tiigi d72c19071a imageutil: use leases on fetching config
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-05-28 14:39:05 -07:00
Tonis Tiigi 2dc7a133a2 content: fix setting wrong gc labels on config query
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-18 12:00:09 -07:00
Derek McGowan edd603e4cd Update containerd vendor
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-02-13 11:03:48 -08:00
Tonis Tiigi 97eff70c5e inline remote cache support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-01-21 19:39:15 -08:00
Tonis Tiigi fe71cd39bd imageutil: fix getting schema1 configs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-09-19 14:09:32 -07:00
Derek McGowan 279a144686 Update containerd vendor
Picks up platform changes

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-28 14:26:14 -07:00
Tõnis Tiigi f3b743bd13
Merge pull request #467 from dmcgowan/export-oci-media-types
Support creating and pushing OCI types
2018-06-28 14:46:34 -07:00
Derek McGowan 618f34a1f9 Support creating and pushing OCI types
Adds image and oci exporter option "oci-mediatypes"
Ensures that the images created in the content store
have the correct type which matches the manifest.
Sets the correct media type on the descriptor in push from
reading the type specified in the manifest.
Removes use of distribution manifest packages.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-06-28 11:23:35 -07:00
Tonis Tiigi f9383c48a3 source: image source platform support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-25 10:39:17 -07:00
Akihiro Suda 47a00b84fe update containerd library
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-06-08 21:44:17 +09:00
Ian Campbell ef1a79a1c1 Allow caller to specify platform for use with imagemetaresolver.
This needs to be plumbed through to `imageutil.Config` which is also used from
the `containerimage` package. Since that is used in the context of a Worker the
`platforms.Default()` (corresponding to the platform of the worker itself) is
appropriate so apply a default at that level.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-05-24 11:23:35 +01:00
Akihiro Suda 126f91dea7 update containerd (binary: v1.0.3, library: Apr 3, 2018)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-04-04 17:22:20 +09:00
Tonis Tiigi d0f8125af9 imageutil: avoid pulling unwanted manifests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-12 15:02:11 -08:00
Akihiro Suda c71a1ca768 bump up containerd (v1.0.0-beta.3)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-11-13 05:36:59 +00:00
Tonis Tiigi 56e1f71e6b cache: allow pushing exported cache
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-13 15:37:55 -07:00
Tonis Tiigi 451be0e504 exporter: revert to distribution/schema2, add push support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-10-13 10:08:47 -07:00
Tonis Tiigi 05c39612d8 dockerfile: resolve to digest ref in frontend
Less hacky than keeping the cached resolver.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-22 15:49:37 -07:00
Tonis Tiigi f20e4d2577 frontend: allow config detection from manifest list
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-15 18:52:09 -07:00
Tonis Tiigi fd0f7ee2a8 frontend: fix dockerfile frontend resolver
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-15 18:09:28 -07:00
Tonis Tiigi b36df4738a frontend: add more commands to dockerfile
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-15 18:09:28 -07:00
Akihiro Suda 279940b60d vendor containerd d1e11f17ec7b325f89608dd46c128300b8727d50
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-09-14 19:06:31 +00:00
Tonis Tiigi 6d335c3c86 imagesource: share config resolver with frontend
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-09-05 13:11:27 -07:00