Commit Graph

306 Commits (37763ec29f6226a3584ba1a87eb5347e1d40c43c)

Author SHA1 Message Date
Kohei Tokunaga 17105bfc50 monitor: resolve paths arguments in client
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Justin Chadwell c2e11196dd controller: handle attestation options across api
We can perform all attestation processing, handling how the sbom and
provenance arguments interact on the client, while applying defaults on
the server.

Additionally, this allows us to start pulling fields out of CommonOpts.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell 0b8f0264b0 controller: move image id file writing to client
Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell 5c31d855fd controller: return solve response through api
Now clients can access the result of the solve, specifically the image
id output. This is a useful refactor, as well as being required if we
want to allow bake to invoke through the controller api.

This also allows us to remove the quiet option from the API, since we
can compute the required progress type outside of the controller, and
can print the image id from the result of the solve.

As a follow-up, we should also be able to remove the image id file
output from the controller api, now that the client has access to it.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell 90d7fb5e77 controller: strongly type the controller api
Strongly typing the API allows us to perform all command line parsing
fully on the client-side, where we have access to the client local
directory and all the client environment variables, which may not be
available on the remote server.

Additionally, the controller api starts to look a lot like
build.Options, so at some point in the future there may be an
oppportunity to merge the two, which would allow both build and bake to
execute through the controller, instead of needing to maintain multiple
code paths.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell ed4fd965ff
Merge pull request from jedevc/remote-controller-fixes
Improvements for remote controller code
CrazyMax 598f1f0a62
build: print instance being used
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax 89d99b1694
inspect: display labels and gc policies
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Justin Chadwell 56b9e785e5 build: don't kill remote controller after build
We don't know if other builds might be running, etc, so we should allow
the server to decide when to exit.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell 4bdf98cf20 lint: ban fmt.Errorf in preference of errors.Errorf
Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell a9fd128910 controller: move controllers out of commands into separate package
Signed-off-by: Justin Chadwell <me@jedevc.com>
Kohei Tokunaga a27b8395b1 Enable to run build and invoke in background
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
CrazyMax 6cd0c11ab1
rm: do not check for context builders when removing inactive
This change has been introduced in e7b5ee7518
but we should not check context builders when removing inactive
ones.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Sebastiaan van Stijn b8285c17e6
docs: update anchor links
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
CrazyMax 8c47277141
store: set nodegroup last activity
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Justin Chadwell 25aa893bad bake: add attests field
Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell b270a20274 build: add attests flag and sbom/provenance shorthands
Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell 0c0c9a0030 chore: sort buildOptions alphabetically
Signed-off-by: Justin Chadwell <me@jedevc.com>
CrazyMax e7b5ee7518
mutualize builder logic
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax 63073b65c0
dockerutil pkg to manage docker api client and context
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax 4788eb24ab
imagetools: set default repo ref on creation if nil
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax eba49fdefd
Merge pull request from crazy-max/docker-api
docker api: use helper to parse context docker endpoint metadata
CrazyMax e66410b932
Merge pull request from jedevc/bake-group-recurse
bake: recursively resolve groups
CrazyMax 3f6a5ab6ba
kubernetes: move context config logic to its own pkg
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
David Karlsson be3b41acc6 docs: refactored file and directory structure
Signed-off-by: David Karlsson <david.karlsson@docker.com>
Justin Chadwell a50e89c38e progress: explicitly fail if tty requested but not available
The NewPrinter function is mostly borrowed from buildkit. However, at
some point, it seems that the implementations drifted.

This patch updates buildx to be more similar in behavior to it's
buildkit counterpart, specifically, it will explicitly fail if a TTY
output is requested using "--progress=tty", but the output is not
available.

To gracefully fallback to plain progress in this scenario,
"--progress=plain" is required.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell a4c3efe783 lint: add nolintlint and fix violations
We should be able to detect nolint comments that point to linters that
are disabled (such as with the removed structcheck).

Signed-off-by: Justin Chadwell <me@jedevc.com>
CrazyMax 58d3a643b9
Merge pull request from crazy-max/bake-print
bake: do not fail printing definition if instance unavailable
CrazyMax c53b0b8a12
bake: do not fail printing definition if instance unavailable
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Justin Chadwell 966c4d4e14 invoke: load defaults from image config
If user does not specify image certain container parameters, we can load
them from the exporter metadata.

Additionally, we introduce a new "default" value for the --invoke flag,
that keeps all of the default parameters (since cobra does not have an
easy way of accepting an optional flag argument).

Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell 77b33260f8 bake: recursively resolve groups
Groups that contained other groups were not recursively resolved by
ReadTargets, which prevented output from --print from being useable as a
self-contained bake file.

This patch ensures that all groups that are referenced inside the bake
file are actually defined under the groups field. This has required a
substantial refactor, as previously only a single group was returned
from ReadTargets, notably, returning a map of groups, instead of a
slice.

This does introduce a small behavior change to the behavior of --print -
while previously, passing a group name to bake would return all the
targets of that group back as the default group, now only the name of
that group will be inserted into the default group, keeping the original
group intact. The impact of this can be observed in some of the changes
to the bake_test.go file.

Signed-off-by: Justin Chadwell <me@jedevc.com>
zhyon404 2f47838ea1 return di.Err when it's not nil
Signed-off-by: zhyon404 <zhyong4@gmail.com>
Tõnis Tiigi ed00243a0c
Merge pull request from jedevc/inspect-buildkit-version
inspect: add buildkit version information to command output
Justin Chadwell 7f9cad1e4e buildx: prevent duplicate "failed to find driver" message
Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell 8f0d9bd71f inspect: add buildkit version information to command output
Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell 441853f189
Merge pull request from jedevc/consistent-experimental-help
Consistent experimental help
Justin Chadwell d01d394a2b build: ensure consistent help messages for experimental cli help
Append an [experimental] tag to the end of each experimental command to
highlight that these are experimental options. Square brackets are used
instead of parentheses as parentheses are already in use to highlight
examples and defaults.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell 17d4369866 create: improve interface when attempting to create docker driver
Previously, the help information for buildx indicated that users could
create a new instance of the docker driver - which is explicitly
something we don't support, driver of this form are automatically
derived from the available list of docker contexts.

This patch ensures that don't have AllowsInstance set will not appear in
the help text, and additionally provide a new more specific error
message instead of the generic "failed to find driver". This should help
point users in the correct direction.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Tonis Tiigi fb5e1393a4 commands: use buildx env for experimental opt-in
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Tonis Tiigi c834ba1389 add formatting support to print function
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Tonis Tiigi eefa8188e1 build: add experimental support for print flag
Print flag can be used to make additional information
requests about the build and print their results.

Currently Dockerfile supports: outline, targets, subrequests.describe

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Tõnis Tiigi 17dc0e1108
Merge pull request from ktock/invoke-messages
invoke: add messages
CrazyMax a7753ea781
Merge pull request from jedevc/prune-all-help
docs: add correct definition of prune --all flag
Justin Chadwell 12a6eb5b22 docs: add correct definition of prune --all flag
The previous definition was the same as the docker images prune command
and referenced dangling images, which isn't what the command does. This
commit brings the command description more inline with the buildctl
definition.

Additionally, add some more description of what the various flags do in
our reference pages.

Signed-off-by: Justin Chadwell <me@jedevc.com>
CrazyMax 74b21258b6
Merge pull request from jedevc/prune-filter-until
Fix prune `--filter until=<duration>` option
Kohei Tokunaga 2f9d46ce27 invoke: add messages
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Justin Chadwell 406799eb1c prune: cleanup variable names for clarity
Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell ef0cbf20f4 buildx: warn on editing nodes
Previously, editing nodes to contain a new set of driver options or
config files was unsupported, and silently dropping them. In this patch,
we update with these, as well as add a new warning message that any new
options may not taken into account until the builder restarts (which
may apply to the flags, platforms and endpoints as well).

Signed-off-by: Justin Chadwell <me@jedevc.com>
CrazyMax 0defb614a4
docker api: use helper to parse context docker endpoint metadata
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Justin Chadwell febcc25d1a prune: fix filter until option
Previously, when specifying the filter option with the until value, no
cache would be cleaned, preventing users from clearing by time. This bug
arises from passing the until field through into buildkit, where, on
filtering, a non-existent field returns false for a match.

The fix is simple, as we build up our list of filters to pass to
buildkit, we skip over the until key, so create a valid list of filters
for buildkit.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell 3f5974b7f9 buildx: forbid mismatched drivers
This patch reorders+refactors the runCreate function to ensure that we
can detect and notify the user in the scenario that the user attempts to
combine multiple drivers in a single builder, which is an unsupported
scenario.

Previously, we would just overwrite the previous builder with the new
driver, potentially invalidating the already existing nodes.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell 7ab3dc080b kubernetes: error about unused endpoint argument
Signed-off-by: Justin Chadwell <me@jedevc.com>
Tõnis Tiigi 1f6612b118
Merge pull request from jedevc/imagetools-multiple-repositories
Imagetools multiple repositories
Tõnis Tiigi c1fbebe73f
Merge pull request from jedevc/init-builder-errors
buildx: log errors in initializing builders
CrazyMax 7ef679d945
bake(compose): allow dot in target name
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax e27a5966ef
check context builder endpoint
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Justin Chadwell 4ecca34a42 imagetools: give imagetools create a progress bar
Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell 37ca8631f9 imagetools: copy manifests between repositories
Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell d3412f1039 imagetools: refactor combining repository logic
This patch modifies the existing combining code in imagetools create to
provide better support for multiple repositories down the road.
Specifically, the code should no longer rely on a single repository
being used for all sources and tags, and should resolve descriptors in
their relevant repositories.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell 86825a95ce buildx: rollback configuration if create fails
This builds on the added warnings from initialized builders, now
erroring the command, and additionally attempting to revert to the
previous configuration.

To preserve the previous configuration, we add a deep Copy() function to
the NodeGroup and Node so that we can easily restore it later if we
encounter a failure.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell 309c49413c buildx: log errors in initializing builders
Previously, errors within the driver config would not be reported to the
user until they tried to use the driver, even though they are easily
accessible from the node group info.

This patch reports these errors (but will not fail because of them,
since the data is already saved) - this should help improve
debuggability of some of the more complex drivers, and prevent error
messages being suppressed.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Justin Chadwell 3b4780ef19 vendor: bump buildkit to master
Signed-off-by: Justin Chadwell <me@jedevc.com>
Tõnis Tiigi a8bb25d1b5
Merge pull request from ktock/invoke
Add `--invoke` option to launch a container from the build result
Kohei Tokunaga 387e1ecca6 Add `--invoke` option to launch a container from the build result
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
Tõnis Tiigi 0295555a5a
Merge pull request from crazy-max/ls-progress
ls: move builder/node status error msg below table
Tõnis Tiigi 71e4a39ae9
Merge pull request from jedevc/remote-driver-connhelpers
Remote driver connhelpers
Justin Chadwell 1eff9310f8 remote: add additional connhelpers to buildx
Signed-off-by: Justin Chadwell <me@jedevc.com>
CrazyMax 064bd92583
ls: move builder/node status error msg below table
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax d611bbe609
rm: display name of removed builder
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax 1e71a3ffa7
rm: disallow removing context builders
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax 69d95cc847
create: warn if instance name already exists as context builder
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax 78ae826d74
ls: dedup instances from store and context
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax 8257a04a7d
create: load default buildkit config if none specified
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax 57156ee95c
ls: adds fallback if buildkit version info unimplemented
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax 6e3babc461
ls: display buildkit version of the nodes
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax 92009ed03c
Merge pull request from tonistiigi/closeread-ignore
root: ignore SSH CloseRead warning
Tonis Tiigi 72758fef22 root: ignore SSH CloseRead warning
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Justin Chadwell d7e4affe98 Complete remote driver
This patch completes the work started in creating a remote driver:

- Renames the env driver to the remote driver (an alternative suggestion
  that should be more user-friendly)
- Adds support for TLS to encrypt connections with buildkitd
- Fixes outstanding review comments
- Reworks the buildx create command endpoint construction to be clearer
  and include better support for this new driver.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Batuhan Apaydın 3dc83e5dd8 feat: env driver
Co-authored-by: Furkan Türkal <furkan.turkal@trendyol.com>
Signed-off-by: Batuhan Apaydın <batuhan.apaydin@trendyol.com>
CrazyMax 95086cf641
cli: fix standalone command behavior
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Zsolt 4a226568a0 Fix tolerations not parsing its options correctly, add tests
Signed-off-by: Zsolt <zsolt.szeberenyi@figured.com>
Justin Chadwell 04b44b3a89 imagetools: respect --builder flag
The --builder flag was being ignored by imagetools because of pointer
problems. Essentially, because the root cmds aren't parsed immediately,
we need to pass a pointer to the builder string so that it can be
updated before the RunE function gets called.

Signed-off-by: Justin Chadwell <me@jedevc.com>
CrazyMax 082c83b825
inspect: fix printing of driver options
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax 1ca30a58c2
Merge pull request from developer-guy/feature/1001
feat: printing driver options
Batuhan Apaydın da4c27e9af
feat: printing driver options
Signed-off-by: Batuhan Apaydın <batuhan.apaydin@trendyol.com>
Sebastiaan van Stijn fc7ba75fd7
Remove uses of deprecated io/ioutil
The package has been deprecated since Go 1.16: https://go.dev/doc/go1.16#ioutil

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Tonis Tiigi c255c04eed bake: restore consistent output for metadata
Metadata formatting should not depend on the number
of targets.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Tõnis Tiigi 6efcee28d5
Merge pull request from crazy-max/vendor-docker-cli
vendor: update docker/cli to 8667ccd
CrazyMax 971b5d2b73
vendor: update docker/cli to 8667ccd
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax 94c5dde85a
docs: small fixes
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Tõnis Tiigi 7af29802d4
Merge pull request from crazy-max/buildinfo-cmd
imagetools inspect: add --format flag
Tõnis Tiigi 6ac01ec9ac
Merge pull request from tonistiigi/bake-context-validation
bake: additional support for named context on remote inputs
CrazyMax 20a55e9184
imagetools inspect: multi-platform support
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
CrazyMax 6c56109083
imagetools inspect: add --format flag
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Tonis Tiigi 280c008f81 bake: make named contexts relative to remote bake input
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Tonis Tiigi b5bf28d722 bake: use better error in named contexts not supported
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Tonis Tiigi e98a476dc8 set build-args from docker proxy configuration
For backward compatibility with docker build.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
CrazyMax 230dfa96a3
docs: built-in build args
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Tõnis Tiigi eef6deb7c2
Merge pull request from tonistiigi/no-cache-filter
build: add no-cache-filter
Tõnis Tiigi e5f590a7fa
Merge pull request from crazy-max/metadata-output
build: enhance metadata json output