Commit Graph

149 Commits (a3293cdaaaab348fe2e532de2d7883fbc5a6cdff)

Author SHA1 Message Date
Justin Chadwell 589d4e4cf5 build: use buildkit's gitutil package to detect remote files
BuildKit's gitutil package behaves slightly differently than moby's
urlutil, so we should rely on BuildKit's gitutil when detecting URLs to
avoid cases of accidentally producing invalid build requests that can
confuse users.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell c6cdcb02cf build: replace removed buildkit BUILDINFO field
Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell 5d4223e4f8 build: move SOURCE_DATE_EPOCH parsing into option generation
This allows the build package code to become more generic, and also
ensures that when the environment variables are not propogated (in the
case of the remote controller), that we can still correctly set
SOURCE_DATE_EPOCH.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Kohei Tokunaga e8f55a3cf7 monitor: Enable to exec into the container
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2 years ago
Justin Chadwell c2e11196dd controller: handle attestation options across api
We can perform all attestation processing, handling how the sbom and
provenance arguments interact on the client, while applying defaults on
the server.

Additionally, this allows us to start pulling fields out of CommonOpts.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell 0b8f0264b0 controller: move image id file writing to client
Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell 5c31d855fd controller: return solve response through api
Now clients can access the result of the solve, specifically the image
id output. This is a useful refactor, as well as being required if we
want to allow bake to invoke through the controller api.

This also allows us to remove the quiet option from the API, since we
can compute the required progress type outside of the controller, and
can print the image id from the result of the solve.

As a follow-up, we should also be able to remove the image id file
output from the controller api, now that the client has access to it.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell 90d7fb5e77 controller: strongly type the controller api
Strongly typing the API allows us to perform all command line parsing
fully on the client-side, where we have access to the client local
directory and all the client environment variables, which may not be
available on the remote server.

Additionally, the controller api starts to look a lot like
build.Options, so at some point in the future there may be an
oppportunity to merge the two, which would allow both build and bake to
execute through the controller, instead of needing to maintain multiple
code paths.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell 1c6060f27d
Merge pull request #1612 from cpuguy83/env_no_provenance
Add env var to disable default attestations
2 years ago
Brian Goff bc9cb2c66a Add env var to disable default attestations
For certain cases we need to build with `--provenance=false`.
However not all build envs (especially in the OSS ethos) have the latest
buildx so just blanket setting `--provenance=false` will fail in these
cases.

Having an env var allows people to set the value without having to worry
about if the buildx version has the `--provenance` flag.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2 years ago
Justin Chadwell cb94298a02
Merge pull request #1296 from ktock/monitor-list
monitor: Enable to run build and invoke in background
2 years ago
Tonis Tiigi c33b310b48
build: fix multi-node merge to read descriptor from result
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2 years ago
Kohei Tokunaga a27b8395b1 Enable to run build and invoke in background
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2 years ago
Tõnis Tiigi b1b4e64c97
Merge pull request #1504 from jedevc/fix-1497
Refactor BuildWithResultHandler to simplify concurrency usage
2 years ago
Tonis Tiigi be55b41427
build: mark capabilities request as internal
So it doesn't show up in the History API.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2 years ago
Justin Chadwell 43a748fd15 build: error when using docker exporter and attestations
Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell 15a80b56b5 build: avoid compatability error when attestations disabled
We should avoid erroring with attestations support compatability errors
when a user has specified --provenance=false.

A user may wish to enable --provenance=false that works across buildkit
versions, but currently it will fail on old versions - this patch fixes
this, to silently ignore the provenance flag for this check if it's set
to disabled.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell 8b7aa1a168 build: create error group per opt
Using the syncronization primitive, we can avoid needing to create a
separate wait group.

This allows us to sidestep the issue where the wait group could be
completed, but the build invocation functions had not terminated - if
one of the functions was to terminate with an error, then it was
possible to encounter a race condition, where the result handling code
would begin executing, despite an error.

The refactor to use a separate error group which more elegantly handles
the concept of function returns and errors, ensures that we can't
encounter this issue.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell 1180d919f5 build: reorder error group funcs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell 347417ee12 build: use copy for BuildWithResultHandler loop vars
Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell 60c9cf74ce
vendor: update buildkit to v0.11.0-rc4
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Tõnis Tiigi ff6754eb04
Merge pull request #1456 from jedevc/oci-layout-reference-parsing
build: refactor reference parsing for oci image layouts
2 years ago
Akihiro Suda 0e6f5a155e
Propagate SOURCE_DATE_EPOCH from the client env
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2 years ago
Justin Chadwell e584c6e1a7 build: check error from toSolveOpt before adding FrontendAttrs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Tõnis Tiigi 64e4c19971
Merge pull request #1477 from crazy-max/git-wsl
build: lookup the right git binary on WSL
2 years ago
CrazyMax 0d1fea8134
build: warn if git operation fails
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
Justin Chadwell a8f689c223 build: forward all build opts everywhere
All build options should be passed everywhere - the frontend and the
backend of buildkit should both be able to see all attestations, as well
as all other opts: e.g. epoch settings, and no-cache.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell 70682b043e build: refactor reference parsing for image layouts
We allow any valid image reference format for the oci-layout, not just
limiting to name@digest, we additionally allow images of the form
name:tag@digest now.

The name of the reference is used to find the local directory to lookup
the store in, while the tag and digest are attached to a random identity
to generate the dummy reference sent to the oci-layout context.

This separation of the target to replace and the value to replace it
with ensures that any tag or digest set in the client is properly sent
across to the server. The tag is used when a digest was not specified,
and it is resolved in the context of the local directory before being
sent, using the same helpers as we use for the local cache expoter.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
CrazyMax 6ad5e2fcf3
build: set provenance vcs details
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
Tõnis Tiigi abc8121aa8
Merge pull request #1457 from jedevc/add-hosts-length-check
build: don't set add-hosts option if empty
2 years ago
Justin Chadwell 98049e7eda build: don't set add-hosts option if empty
This looks like an oversight, all of the other options have similar
checks. This can interfere with generated provenance where "add-hosts"
will be marked as an argument to the build, even though it's not
actually being utilized.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell f0262dd10e build: add attestations to build options
Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell f8b673eccd build: pass attestation attributes to build request
Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
CrazyMax e7b5ee7518
mutualize builder logic
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
CrazyMax 63073b65c0
dockerutil pkg to manage docker api client and context
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
CrazyMax 49d46e71de
build: skip "host-gateway" validation with moby driver
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
Tõnis Tiigi ac85f590ba
Merge pull request #1324 from jedevc/invoke-defaults
invoke: load defaults from image config
2 years ago
David Gageot 8fd34669ed
Fix assignment to nil map
Signed-off-by: David Gageot <david.gageot@docker.com>
2 years ago
Justin Chadwell 966c4d4e14 invoke: load defaults from image config
If user does not specify image certain container parameters, we can load
them from the exporter metadata.

Additionally, we introduce a new "default" value for the --invoke flag,
that keeps all of the default parameters (since cobra does not have an
easy way of accepting an optional flag argument).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
CrazyMax 25ceb90678
Merge pull request #1294 from crazy-max/s3-cache
s3 cache client-side support
2 years ago
Christian Dupuis e3c91c9d29
Add git provenance labels
as per #1290

Signed-off-by: Christian Dupuis <cd@atomist.com>
2 years ago
CrazyMax 57d22a7bd1
s3 cache client-side support
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
Tonis Tiigi 18dbde9ed6 build: update outline fallback image
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2 years ago
Tõnis Tiigi da1f4b8496
Merge pull request #1100 from tonistiigi/print-outline
Build: Support for printing outline/targets of the current build
2 years ago
Tonis Tiigi c834ba1389 add formatting support to print function
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2 years ago
Tonis Tiigi cab437adef build: add fallback to outline requests if not supported by frontend
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2 years ago
Tonis Tiigi eefa8188e1 build: add experimental support for print flag
Print flag can be used to make additional information
requests about the build and print their results.

Currently Dockerfile supports: outline, targets, subrequests.describe

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2 years ago
Tonis Tiigi 75ddc5b811 build: fix issues with leaving invoke containers running
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2 years ago
Kohei Tokunaga 2f9d46ce27 invoke: add messages
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2 years ago
Tõnis Tiigi 1f6612b118
Merge pull request #1137 from jedevc/imagetools-multiple-repositories
Imagetools multiple repositories
2 years ago