Commit Graph

628 Commits (bc07b2b81b1c6a62d29981ac564b16a15ce2bfa7)

Author SHA1 Message Date
Edgar Lee 334c62f3b7 Transform relative mountpoints for exec mounts in the executor
Signed-off-by: Edgar Lee <edgarl@netflix.com>
(cherry picked from commit 9aeed7b193)
2021-05-26 19:45:29 -07:00
Tonis Tiigi a1094c61f9 contenthash: include basename in content checksum for wildcards
While we generally ignore the basename in this layer, for wildcards
there in no other place to add the basename to the checksum as they
can not be resolved earlier. Before the basename that was in the
checksum was the wildcard itself, so if the wildcard remained same,
content remained same but the file where wildcard pointed to was
renamed, the cache was not invalidated.

Unfortunately, this change breaks cache for all copy commands that
use a wildcard.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 1982e1e285)
2021-04-21 23:26:58 -07:00
Tonis Tiigi ce6dc4bd0d
don't commit cache mounts on error
Cache mount instances are shared between multiple vertextes/builds
so if one of the cloned instance gets committed reference count
will get corrupted as other parts of the code still see reference as
mountable.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 3660e5f9c8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-21 20:25:04 +01:00
Tibor Vass c7267e12cf
dockerfile/docs: fix frontend image tags
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 1218e37c23)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-21 20:19:44 +01:00
Sebastiaan van Stijn 5e11c07cc7
frontend/dockerfile: add RunCommand.FlagsUsed field
The FlagsUsed contains a list of flags that were used, which allows the classic
(non-buildkit) builder in dockerd to produce an error when non-supported options
are used in a Dockerfile.

This is a short-term solution; a more permanent solution will be to keep track
of which version of the Dockerfile syntax is supported, and to have the classic
builder pass the maximum supported version of the syntax.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ebed917e43)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-04 22:23:25 +01:00
Tonis Tiigi 3889ef509c allow nil return from frontend
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-14 17:19:41 -08:00
Tonis Tiigi 9f6172c8bd docs: update external dockerfile docs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-08 21:26:18 -08:00
Sebastiaan van Stijn 4e9bae48e7
Revert "Ensure ENTRYPOINT command has at least one argument"
This reverts commit 174bcf85ef.

This commit attempted to fix a situation where an empty entrypoint
was specified, causing a confusing error when running the image,
however, allowing the entrypoint to be reset should be a valid
use-case, and running such image on docker 20.10 at least
produces an informative error;

    docker build -t foo -<<'EOF'
    FROM busybox
    ENTRYPOINT []
    EOF

Or, to reset a previously set entrypoint:

    docker build -t foo -<<'EOF'
    FROM busybox AS one
    ENTRYPOINT ["/bin/busybox"]

    FROM one AS two
    ENTRYPOINT []
    EOF

If no command is specified for the image above:

    docker run -it --rm foo
    docker: Error response from daemon: No command specified.
    See 'docker run --help'.

Passing a command to run:

    docker run -it --rm foo sh
    /#

Given that this commit resulted in a regression/breaking change
this reverts the commit.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-12-03 13:36:52 +01:00
Tonis Tiigi 2df822c610 dockerfile: fix escaping in release script
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-02 19:02:21 -08:00
Tõnis Tiigi abb3325adf
Merge pull request #1862 from chang-andrew/chang-andrew/main/bk-1119-empty-entrypoint
Ensure ENTRYPOINT command has at least one argument
2020-12-02 13:23:39 -08:00
Andrew Chang 174bcf85ef Ensure ENTRYPOINT command has at least one argument
Signed-off-by: Andrew Chang <chang331006@gmail.com>
2020-12-01 15:05:25 -06:00
Tonis Tiigi e46cfab9fb fix invalid cancellation errors on pull
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-01 00:27:31 -08:00
CrazyMax 1ac6bd4cbc
Fix CI cache
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2020-11-29 06:45:39 +01:00
CrazyMax d754dda292
Refactor frontend for GitHub Actions
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2020-11-26 02:13:18 +01:00
Tonis Tiigi ab9dc87705 dockerfile: allow lowercase dockerfile name
This was supported by the legacy builder moby/moby#10858


Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-18 23:31:26 -08:00
Sebastiaan van Stijn 67bcfe699a
copy containerd.UnknownExitStatus to local const
Copy this const to a local constant to prevent importing the containerd
client in the front-end.

For consistency, I also updated the executor code to use the same const,
although not strictly needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-18 00:11:09 +01:00
Tonis Tiigi 81f74a9c89 dockerfile: rename experimental channel to labs
Experimental name confuses users as backwards compatibility
rules are different for other tools called experimental.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-17 07:42:50 -08:00
Tõnis Tiigi 2aa3e0bd42
Merge pull request #1395 from lugeng/fix-load-metadata
dockerfile build:  fix not exit when meet error in load config metadata
2020-11-17 00:30:06 -08:00
岁丰 adca74632e dockerfile build: return error when meet error in load config metadata
Signed-off-by: genglu <genglu.gl@antfin.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-16 17:16:02 -08:00
Edgar Lee 1240dd7795 Return committed readonly inputs and actives in exec error in MountIDs
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-16 13:21:23 -08:00
Edgar Lee 3ba6cd7bc2 Fix ExecError.EachRef invoking callback with possibly nil solver.Results
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 23:20:41 -08:00
Edgar Lee 4c0ca1719f Fix container release not capturing closure of loop variable
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:58:49 -08:00
Edgar Lee dfaf613996 Parallelize unlazying ref proxy in the gateway forwarder
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:14:05 -08:00
Edgar Lee c33bcd61a2 Rename OutputIDs to MountIDs
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
Edgar Lee bcff7baf60 Allow scratch mounts with gateway.RunContainer
- Plumb default worker by adding GetDefault() to frontend.WorkerInfos
- To avoid cyclic dependency, refactor frontend.WorkerInfos to worker.Infos
- Refactor gateway.NewContainer to share code with llbsolver/ops/exec.go

Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
Edgar Lee 7e1dc9bec1 Refactor to file action indexed outputs
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
Edgar Lee 2d23d0cc43 Fix lint and unit tests for fileopsolver
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
Edgar Lee 7ce58c31ba Plumb op metadata to recreate failed ops with gateway exec
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2020-11-13 22:05:13 -08:00
Tonis Tiigi bdcee17437 executor: change mount to struct
Allows readonly passed cleanly.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:51 -08:00
Tonis Tiigi e3b05289d8 add session injection to remote loading
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-11-02 22:20:44 -08:00
Sebastiaan van Stijn 40e6129775
frontend/dockerfile: use dockerignore package from buildkit
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 11:55:05 +01:00
Sebastiaan van Stijn 31edeb3eab
dockerignore: remove import path enforcement comments
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 11:52:59 +01:00
Sebastiaan van Stijn a9580f569a
Migrate github.com/docker/docker/builder/dockerignore to BuildKit repository
Strategy taken:

    # install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
    brew install git-filter-repo

    cd ~/projects

    # create a temporary clone of docker
    git clone https://github.com/docker/docker.git docker_TEMP
    cd docker_TEMP

    # remove the origin remote (just for safety)
    git remote remove origin

    # create branch to work with
    git checkout -b migrate_dockerignore

    # remove all code, except for builder/dockerignore, and rewrite the files to frontend/dockerfile/dockerignore.
    git filter-repo --force --path builder/dockerignore --path-rename builder/dockerignore:frontend/dockerfile/dockerignore

    # check the results
    tree
    .
    └── frontend
        └── dockerfile
            └── dockerignore
                ├── dockerignore.go
                └── dockerignore_test.go

    3 directories, 2 files

    # go to the target github.com/moby/sys repository
    cd ~/projects/buildkit

    # create a branch to work with
    git checkout -b integrate_dockerignore

    # add the temporary repository as an upstream and make sure it's up-to-date
    git remote add docker_TEMP ~/projects/docker_TEMP
    git fetch docker_TEMP

    # merge the upstream code
    git merge --allow-unrelated-histories --signoff -S docker_TEMP/migrate_dockerignore

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-30 11:51:18 +01:00
Tibor Vass 6b1c950745
Merge pull request #1559 from thaJeztah/fix_trailing_escape
Dockerfile: fix parsing of trailing backslash
2020-10-28 23:54:42 -07:00
Cory Bennett ffd4ab232e gateway exec: add platform and worker constraints to NewContainer api
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-28 03:54:49 +00:00
Sebastiaan van Stijn cc7a3ded6f
Dockerfile: fix parsing of trailing backslash
If the line-continuation marker (`\`) is escaped, it should not be
treated as such, but as a literal backslash.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-26 12:29:05 +01:00
Sebastiaan van Stijn e367fd69c3
TestParseCases(): fix "expected" and "actual" being reversed
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-26 12:27:17 +01:00
Tonis Tiigi cec6dae19d dockerfile: add test for default shell and path
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-24 11:12:15 -07:00
Tonis Tiigi 63856b6a36 dockerfile: set default shell based on OS
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-23 15:30:23 -07:00
Tonis Tiigi ecf070a027 exec: use platform specific default path
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-23 15:30:23 -07:00
Akihiro Suda dda009a58c
Merge pull request #1739 from tonistiigi/empty-layer
clear file mount stubs and fix empty layer cases
2020-10-21 00:42:45 +09:00
Edgar Lee 5eaecb905c
Merge pull request #1731 from coryb/issue-1714
add tty support for runc executor
2020-10-19 14:52:31 -07:00
Tonis Tiigi 17b4ca1edf executor: remove stub files from rootfs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-18 23:39:17 -07:00
Cory Bennett cb12a5642f shutdown container process when context is done
Signed-off-by: Cory Bennett <cbennett@netflix.com>
2020-10-17 06:53:25 +00:00
Lu Jingxiao 059dac1e05 tests: Adding tests for hostname specifying for building
Adding tests to for hostname specifying for image building

Cover: #1301
Signed-off-by: Lu Jingxiao <lujingxiao@huawei.com>
2020-10-14 15:36:47 +08:00
Lu Jingxiao 5e7ae230b2 Add hostname specifying for building
Fix: #1301

Signed-off-by: Lu Jingxiao <lujingxiao@huawei.com>
2020-10-14 14:45:36 +08:00
Tibor Vass 7bdb6592c5
Merge pull request #1693 from tonistiigi/dockerfile-comments
dockerfile: parse comments associated with args and stages
2020-10-13 00:20:49 -07:00
Tonis Tiigi 0a7f6ccf5e dockerfile: parse comments associated with args and stages
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-12 23:11:23 -07:00
Tibor Vass 37bb903a31
Merge pull request #1724 from tonistiigi/subreq
frontend: support for subrequests
2020-10-12 11:43:59 -07:00
Tonis Tiigi b4fad847ac frontend: support for subrequests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-10-07 13:36:51 -07:00