Commit Graph

2806 Commits (0bd43aa1dc505b096f81f9e7cc7a42579185b5ca)

Author SHA1 Message Date
Tonis Tiigi ec756078f3 vendor: update containerd to fa1220fce
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-10 20:12:46 -07:00
Erik Sipsma 3b418db323 llb: Cache DefinitionOp inputs to support memoization during Marshal
llb.State.Marshal uses a vertexCache map (keyed by the llb.Vertex interface
value) to memoize which vertexes it has already visited while marshalling.

However, before this change, llb.DefinitionOp was constructing new pointers for
each value in the return slice of Inputs(), which meant that each input had a
different key in the vertexCache (due to being different pointer values). This
meant no memoization actually occured. I noticed this while using
llb.DefinitionOp with a fairly large graph when my program crashed after using
>16GB of RSS during a call to llb.State.Marshal.

The fix here avoids changing vertexCache as that would impact many other
vertex implementations. Instead, just DefinitionOp.Inputs() is updated to cache
the pointers it creates and pass that cache to its descendents. This fix
resulted in the program that was previously crashing out-of-memory to run
without any perceivable increase in RSS.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-08-10 17:23:04 -07:00
Tõnis Tiigi 279d686fec
Merge pull request #1567 from everpeace/fix-authority-header
Setting `Host` part  of address url to `:authority` pseudo header
2020-08-06 15:43:00 -07:00
Tõnis Tiigi 545532ab0e
Merge pull request #1475 from sipsma/lazy-ref
Add support for lazily-pulled blobs in cache manager.
2020-08-06 12:54:45 -07:00
Tõnis Tiigi d88fb2170b
Merge pull request #1602 from tonistiigi/grpcerrors-typeurl
grpcerrors: use full typeurl registration
2020-08-06 12:48:15 -07:00
Tõnis Tiigi 75c0c1a381
Merge pull request #1629 from AkihiroSuda/podman-connhelper
new connection helper: podman-container://<CONTAINER>
2020-08-06 12:30:56 -07:00
Tonis Tiigi 862387445e grpcerrors: use full typeurl registration
Also switches current types to json.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-06 12:28:48 -07:00
Akihiro Suda 6d14bfd206
Merge pull request #1628 from tonistiigi/connhelper-fix
client: fix connhelper context canceling
2020-08-06 15:57:19 +09:00
Akihiro Suda 8d5e9f8294
new connection helper: podman-container://<CONTAINER>
Similar to `docker-container://` but for Podman containers.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-08-06 15:38:16 +09:00
Tonis Tiigi 9fcd482211 client: fix connhelper context canceling
Dial context is canceled after connection is
established so can’t be used for a long running process.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-05 23:25:13 -07:00
Erik Sipsma 1b30fd146b cache: Remove ImageRef from DescHandlers
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-08-05 17:18:43 -07:00
Erik Sipsma 926ca1804c pull: de-pointer non-optional platform field
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-08-05 17:18:43 -07:00
Erik Sipsma cdcf49fd18 Fix progress in schema1 pull.
Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-08-05 17:18:43 -07:00
Erik Sipsma 55cbd19dec Add support for lazily-pulled blobs in cache manager.
This allows the layers of images to only be pulled if/once they are actually
required.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
2020-08-05 17:18:43 -07:00
Akihiro Suda ba0150f7b2
Merge pull request #1623 from tonistiigi/travis-deploy-debug
travis: replace after_failure with after_script
2020-08-04 12:30:22 +09:00
Tonis Tiigi 22495b75d6 travis: replace after_failure with after_script
after_failure does not get triggered after deploy failure. 

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-03 13:35:57 -07:00
Tõnis Tiigi dd6df88897
Merge pull request #1619 from coryb/execute-status
return exit status from executor Run/Exec
2020-08-03 08:54:20 -07:00
Akihiro Suda 2f5e82529f
Merge pull request #1622 from tonistiigi/resolver-performance
resolver: fix resolver reuse
2020-08-03 16:37:19 +09:00
Cory Bennett 19c0077b49 update container resize events in sequence, also move it out of exit/cancel loop to prevent blocking.
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-08-03 01:37:38 +00:00
Tonis Tiigi 22061b1ec9 resolver: reenable keep-alive
Was disabled with http2 but shouldn’t have been.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-02 09:32:46 -07:00
Tonis Tiigi 5e08a24e88 resolver: make sure authorizer is not regenerated
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-02 09:32:40 -07:00
Akihiro Suda b1db6633bf
Merge pull request #1613 from tonistiigi/env-parsing-fix
buildctl: improve secret parsing
2020-08-02 22:13:54 +09:00
Tonis Tiigi 3278f8d6d7 pull: fix counter for ensuremanifestloaded check
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-08-01 22:08:22 -07:00
Cory Bennett 86e246a874 only warn on resize errors
prevent resize from blocking exit
fix edgecase where kill signal never reaches process

Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-08-01 22:40:57 +00:00
Cory Bennett f781f83a89 fix containerd executor Run/Exec to close container input on eof from stdin
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-08-01 17:21:40 +00:00
Cory Bennett 93344a9d24 remove *pixel from winsize struct, tweak ExitError handling for ctx.Err
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-31 20:22:12 +00:00
Cory Bennett 4b456f17f4 wrap errors from executor Run/Exec to allow access to exit code
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-07-31 19:14:30 +00:00
Shingo Omura 804aa5d21a
setting host part of address as :authority pseudo header.
grpc-go uses a slightly different naming scheme(https://github.com/grpc/grpc/blob/master/doc/naming.md)
This will end up setting rfc non-complient :authority header to address string (e.g. tcp://127.0.0.1:1234).
So, this commit changes to sets right authority header via WithAuthority DialOption.

Signed-off-by: Shingo Omura <everpeace@gmail.com>
2020-07-31 21:10:45 +09:00
Tõnis Tiigi 3f3957d16c
Merge pull request #1594 from TBBle/generate_oci_specs_on_all_platforms
Unify OCI Spec generation across all host platforms
2020-07-30 20:05:13 -07:00
Paul "TBBle" Hampson 603e1d7100 Don't ignore failure to setup networking
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-31 02:49:39 +10:00
Paul "TBBle" Hampson 8cd927c788 Refactor OCI Spec generation to use oci.SpecOpts more
This has the nice side-effect of unifying the mount- and
non-mount-changes made due to processMode and securityMode.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-31 02:49:39 +10:00
Paul "TBBle" Hampson 59edca02f3 Fix linter warning about comment mismatch
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-31 02:49:39 +10:00
Paul "TBBle" Hampson 883c726bd5 Compile containerd worker on Windows
It's not fully-functional at this time, but it now compiles, which means
we are getting nearer to the point where we can start enabling tests.

This will ensure that it _stays_ compilable over time.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-31 02:49:39 +10:00
Paul "TBBle" Hampson 1527525110 Unify OCI Spec generation across all host platforms
There's only a couple of things that are host-platform limited, i.e.,
security and process modes.

Everything else is specific to a target-platform. We can tell if we're
targeting a Linux platform, either on Linux or LCOW, by the presence of
the Linux key in the generated spec.

This doesn't introduce support for LCOW, as we'd need to plumb that down
from the caller. It will probably also need massaging to work with LCOW,
as some of the setup code here is probably incorrect for LCOW, e.g.,
the bind-mounts list may be incorrect.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-31 02:49:23 +10:00
Tõnis Tiigi 7b6dbdb715
Merge pull request #1615 from thaJeztah/update_deps
vendor: update opentracing-go v1.2.0, go-stdlib v1.0.0
2020-07-30 09:17:37 -07:00
Tõnis Tiigi 86d5a6a737
Merge pull request #1610 from TBBle/allow_equal_time_for_cache_entries
Ensure that time has passed between foo0 and foo1
2020-07-30 09:16:43 -07:00
Tõnis Tiigi 5c4fe66e97
Merge pull request #1618 from TBBle/clarify_gateway_test_as_a_build_test
Clarify the gateway 'test' as a build-test
2020-07-30 09:16:18 -07:00
Tõnis Tiigi bc84c830cb
Merge pull request #1617 from thaJeztah/remove_unused_context
executor/oci.GetUser(): remove unused context
2020-07-30 09:15:41 -07:00
Paul "TBBle" Hampson 3132f4d0ed Clarify the gateway 'test' as a build-test
It also doesn't need dockerd added to its container when it builds.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-30 23:12:29 +10:00
Sebastiaan van Stijn f26452a847
executor/oci.GetUser(): remove unused context
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-30 11:31:03 +02:00
Sebastiaan van Stijn 844c40a4de
vendor: update opentracing-go v1.2.0, go-stdlib v1.0.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-30 11:16:27 +02:00
Akihiro Suda 547db0f588
Merge pull request #1614 from tonistiigi/transport-tracing
resolver: enable tracing on default transport
2020-07-30 16:16:19 +09:00
Tonis Tiigi cdf424f677 resolver: enable tracing on default transport
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-29 23:19:52 -07:00
Paul "TBBle" Hampson f4393eaa31 Ensure that time has passed between foo0 and foo1
On Windows, the resolution of time.Now() can be surprisingly low: 15ms
according to the comments in the source of time.Now().

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-07-30 03:21:30 +10:00
Akihiro Suda 594ebbfac0
Merge pull request #1601 from tonistiigi/insecure-fix
push: reenable setting insecure exporter opt
2020-07-29 13:55:36 +09:00
Akihiro Suda 4d27bf48d1
Merge pull request #1612 from tonistiigi/deploy-cache
hack: use caching for master deploy
2020-07-29 13:55:08 +09:00
Akihiro Suda 251f40343f
Merge pull request #1611 from tonistiigi/update-containerd-0728
update containerd vendor and add 1.4-beta2 testing
2020-07-29 12:39:50 +09:00
Tonis Tiigi 5571a1e605 buildctl: improve secret parsing
type=env,env=foo would not work before

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-28 20:06:00 -07:00
Tonis Tiigi 70c2ab7f4d hack: use caching for master deploy
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-28 19:59:17 -07:00
Tonis Tiigi 942c39adfd add testing for containerd 1.4-beta2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-07-28 18:13:15 -07:00