diff --git a/README.md b/README.md index 885f6956..0f8189f1 100644 --- a/README.md +++ b/README.md @@ -32,14 +32,16 @@ Key features: - [Building with buildx](#building-with-buildx) - [Working with builder instances](#working-with-builder-instances) - [Building multi-platform images](#building-multi-platform-images) - - [High-level build options](#high-level-build-options) - [Guides](docs/guides) + - [High-level build options with Bake](docs/guides/bake/index.md) - [CI/CD](docs/guides/cicd.md) - [CNI networking](docs/guides/cni-networking.md) - - [Registry mirror](docs/guides/registry-mirror.md) - - [Resource limiting](docs/guides/resource-limiting.md) - [Using a custom network](docs/guides/custom-network.md) - [Using a custom registry configuration](docs/guides/custom-registry-config.md) + - [OpenTelemetry support](docs/guides/opentelemetry.md) + - [Registry mirror](docs/guides/registry-mirror.md) + - [Remote builder](docs/guides/remote-builder.md) + - [Resource limiting](docs/guides/resource-limiting.md) - [Reference](docs/reference/buildx.md) - [`buildx bake`](docs/reference/buildx_bake.md) - [`buildx build`](docs/reference/buildx_build.md) @@ -319,26 +321,7 @@ cross-compilation helpers for more advanced use-cases. ## High-level build options -Buildx also aims to provide support for high-level build concepts that go beyond -invoking a single build command. We want to support building all the images in -your application together and let the users define project specific reusable -build flows that can then be easily invoked by anyone. - -BuildKit efficiently handles multiple concurrent build requests and -de-duplicating work. The build commands can be combined with general-purpose -command runners (for example, `make`). However, these tools generally invoke -builds in sequence and therefore cannot leverage the full potential of BuildKit -parallelization, or combine BuildKit’s output for the user. For this use case, -we have added a command called [`docker buildx bake`](docs/reference/buildx_bake.md). - -The `bake` command supports building images from compose files, similar to -[`docker-compose build`](https://docs.docker.com/compose/reference/build/), -but allowing all the services to be built concurrently as part of a single -request. - -There is also support for custom build rules from HCL/JSON files allowing -better code reuse and different target groups. The design of bake is in very -early stages and we are looking for feedback from users. +See [`docs/guides/bake/index.md`](docs/guides/bake/index.md) for more details. # Contributing diff --git a/docs/guides/bake/build-contexts.md b/docs/guides/bake/build-contexts.md new file mode 100644 index 00000000..e6ece739 --- /dev/null +++ b/docs/guides/bake/build-contexts.md @@ -0,0 +1,77 @@ +--- +title: "Defining additional build contexts and linking targets" +keywords: build, buildx, bake, buildkit, hcl +--- + +In addition to the main `context` key that defines the build context each target +can also define additional named contexts with a map defined with key `contexts`. +These values map to the `--build-context` flag in the [build command](https://docs.docker.com/engine/reference/commandline/buildx_build/#build-context). + +Inside the Dockerfile these contexts can be used with the `FROM` instruction or `--from` flag. + +The value can be a local source directory, container image (with `docker-image://` prefix), +Git URL, HTTP URL or a name of another target in the Bake file (with `target:` prefix). + +## Pinning alpine image + +```dockerfile +# syntax=docker/dockerfile:1 +FROM alpine +RUN echo "Hello world" +``` + +```hcl +# docker-bake.hcl +target "app" { + contexts = { + alpine = "docker-image://alpine:3.13" + } +} +``` + +## Using a secondary source directory + +```dockerfile +# syntax=docker/dockerfile:1 +FROM scratch AS src + +FROM golang +COPY --from=src . . +``` + +```hcl +# docker-bake.hcl +target "app" { + contexts = { + src = "../path/to/source" + } +} +``` + +## Using a result of one target as a base image in another target + +To use a result of one target as a build context of another, specity the target +name with `target:` prefix. + +```dockerfile +# syntax=docker/dockerfile:1 +FROM baseapp +RUN echo "Hello world" +``` + +```hcl +# docker-bake.hcl +target "base" { + dockerfile = "baseapp.Dockerfile" +} + +target "app" { + contexts = { + baseapp = "target:base" + } +} +``` + +Please note that in most cases you should just use a single multi-stage +Dockerfile with multiple targets for similar behavior. This case is recommended +when you have multiple Dockerfiles that can't be easily merged into one. diff --git a/docs/guides/bake/compose-xbake.md b/docs/guides/bake/compose-xbake.md new file mode 100644 index 00000000..347337a2 --- /dev/null +++ b/docs/guides/bake/compose-xbake.md @@ -0,0 +1,125 @@ +--- +title: "Extension field with Compose" +keywords: build, buildx, bake, buildkit, compose +--- + +[Special extension](https://docs.docker.com/compose/compose-file/#extension) +field `x-bake` can be used in your compose file to evaluate fields that are not +(yet) available in the [build definition](https://docs.docker.com/compose/compose-file/build/#build-definition). + +```yaml +# docker-compose.yml +services: + addon: + image: ct-addon:bar + build: + context: . + dockerfile: ./Dockerfile + args: + CT_ECR: foo + CT_TAG: bar + x-bake: + tags: + - ct-addon:foo + - ct-addon:alp + platforms: + - linux/amd64 + - linux/arm64 + cache-from: + - user/app:cache + - type=local,src=path/to/cache + cache-to: type=local,dest=path/to/cache + pull: true + + aws: + image: ct-fake-aws:bar + build: + dockerfile: ./aws.Dockerfile + args: + CT_ECR: foo + CT_TAG: bar + x-bake: + secret: + - id=mysecret,src=./secret + - id=mysecret2,src=./secret2 + platforms: linux/arm64 + output: type=docker + no-cache: true +``` + +```console +$ docker buildx bake --print +``` +```json +{ + "group": { + "default": { + "targets": [ + "aws", + "addon" + ] + } + }, + "target": { + "addon": { + "context": ".", + "dockerfile": "./Dockerfile", + "args": { + "CT_ECR": "foo", + "CT_TAG": "bar" + }, + "tags": [ + "ct-addon:foo", + "ct-addon:alp" + ], + "cache-from": [ + "user/app:cache", + "type=local,src=path/to/cache" + ], + "cache-to": [ + "type=local,dest=path/to/cache" + ], + "platforms": [ + "linux/amd64", + "linux/arm64" + ], + "pull": true + }, + "aws": { + "context": ".", + "dockerfile": "./aws.Dockerfile", + "args": { + "CT_ECR": "foo", + "CT_TAG": "bar" + }, + "tags": [ + "ct-fake-aws:bar" + ], + "secret": [ + "id=mysecret,src=./secret", + "id=mysecret2,src=./secret2" + ], + "platforms": [ + "linux/arm64" + ], + "output": [ + "type=docker" + ], + "no-cache": true + } + } +} +``` + +Complete list of valid fields for `x-bake`: + +* `cache-from` +* `cache-to` +* `no-cache` +* `no-cache-filter` +* `output` +* `platforms` +* `pull` +* `secret` +* `ssh` +* `tags` diff --git a/docs/guides/bake/file-definition.md b/docs/guides/bake/file-definition.md new file mode 100644 index 00000000..51899515 --- /dev/null +++ b/docs/guides/bake/file-definition.md @@ -0,0 +1,363 @@ +--- +title: "Bake file definition" +keywords: build, buildx, bake, buildkit, hcl, json, compose +--- + +`buildx bake` supports HCL, JSON and Compose file format for defining build +groups and targets. It looks for build definition files in the current +directory in the following order: + +* `docker-compose.yml` +* `docker-compose.yaml` +* `docker-bake.json` +* `docker-bake.override.json` +* `docker-bake.hcl` +* `docker-bake.override.hcl` + +A target reflects a single docker build invocation with the same options that +you would specify for `docker build`. A group is a grouping of targets. + +Multiple files can include the same target and final build options will be +determined by merging them together. + +> **Note** +> +> In the case of compose files, each service corresponds to a target. + +A group can specify its list of targets with the `targets` option. A target can +inherit build options by setting the `inherits` option to the list of targets or +groups to inherit from. + +## HCL definition + +HCL definition file is recommended as its experience is more aligned with buildx UX +and also allows better code reuse, different target groups and extended features. + +```hcl +# docker-bake.hcl +variable "TAG" { + default = "latest" +} + +group "default" { + targets = ["db", "webapp-dev"] +} + +target "webapp-dev" { + dockerfile = "Dockerfile.webapp" + tags = ["docker.io/username/webapp:${TAG}"] +} + +target "webapp-release" { + inherits = ["webapp-dev"] + platforms = ["linux/amd64", "linux/arm64"] +} + +target "db" { + dockerfile = "Dockerfile.db" + tags = ["docker.io/username/db"] +} +``` + +Complete list of valid target fields: + +* `args` +* `cache-from` +* `cache-to` +* `context` +* `contexts` +* `dockerfile` +* `inherits` +* `labels` +* `no-cache` +* `no-cache-filter` +* `output` +* `platform` +* `pull` +* `secrets` +* `ssh` +* `tags` +* `target` + +## JSON definition + +```json +{ + "variable": { + "TAG": { + "default": "latest" + } + }, + "group": { + "default": { + "targets": [ + "db", + "webapp-dev" + ] + } + }, + "target": { + "webapp-dev": { + "dockerfile": "Dockerfile.webapp", + "tags": [ + "docker.io/username/webapp:${TAG}" + ] + }, + "webapp-release": { + "inherits": [ + "webapp-dev" + ], + "platforms": [ + "linux/amd64", + "linux/arm64" + ] + }, + "db": { + "dockerfile": "Dockerfile.db", + "tags": [ + "docker.io/username/db" + ] + } + } +} +``` + +Same list of target fields as [HCL definition](#hcl-definition) are available. + +## Compose file + +```yaml +# docker-compose.yml +services: + webapp-dev: &dev + build: + dockerfile: Dockerfile.webapp + tags: + - docker.io/username/webapp:latest + + webapp-release: + <<: *dev + build: + x-bake: + platforms: + - linux/amd64 + - linux/arm64 + + db: + image: docker.io/username/db + build: + dockerfile: Dockerfile.db +``` + +> **Limitations** +> +> Bake uses the [compose-spec](https://docs.docker.com/compose/compose-file/) to +> parse a compose file. Some fields are not (yet) available, but you can use +> the [special extension field `x-bake`](compose-xbake.md). +> +> `inherits` service field is also not supported. Use [YAML anchors](https://docs.docker.com/compose/compose-file/#fragments) +> to reference other services. +> +> Specifying variables or global scope attributes is not yet supported for +> compose files. +{: .warning } + +## Remote definition + +You can also use a remote `git` bake definition: + +```console +$ docker buildx bake "https://github.com/docker/cli.git#v20.10.11" --print +#1 [internal] load git source https://github.com/docker/cli.git#v20.10.11 +#1 0.745 e8f1871b077b64bcb4a13334b7146492773769f7 refs/tags/v20.10.11 +#1 2.022 From https://github.com/docker/cli +#1 2.022 * [new tag] v20.10.11 -> v20.10.11 +#1 DONE 2.9s +{ + "group": { + "default": { + "targets": [ + "binary" + ] + } + }, + "target": { + "binary": { + "context": "https://github.com/docker/cli.git#v20.10.11", + "dockerfile": "Dockerfile", + "args": { + "BASE_VARIANT": "alpine", + "GO_STRIP": "", + "VERSION": "" + }, + "target": "binary", + "platforms": [ + "local" + ], + "output": [ + "build" + ] + } + } +} +``` + +As you can see the context is fixed to `https://github.com/docker/cli.git` even if +[no context is actually defined](https://github.com/docker/cli/blob/2776a6d694f988c0c1df61cad4bfac0f54e481c8/docker-bake.hcl#L17-L26) +in the definition. + +If you want to access the main context for bake command from a bake file +that has been imported remotely, you can use the [`BAKE_CMD_CONTEXT` built-in var](hcl-vars-funcs.md#built-in-variables). + +```console +$ cat https://raw.githubusercontent.com/tonistiigi/buildx/remote-test/docker-bake.hcl +``` +```hcl +target "default" { + context = BAKE_CMD_CONTEXT + dockerfile-inline = < [4/4] RUN ls -l && stop: +#8 0.101 total 0 +#8 0.102 -rw-r--r-- 1 root root 0 Jul 27 18:47 bar +#8 0.102 -rw-r--r-- 1 root root 0 Jul 27 18:47 foo +#8 0.102 /bin/sh: stop: not found +``` + +```console +$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test" "https://github.com/docker/cli.git#v20.10.11" --print +#1 [internal] load git source https://github.com/tonistiigi/buildx.git#remote-test +#1 0.429 577303add004dd7efeb13434d69ea030d35f7888 refs/heads/remote-test +#1 CACHED +{ + "target": { + "default": { + "context": "https://github.com/docker/cli.git#v20.10.11", + "dockerfile": "Dockerfile", + "dockerfile-inline": "FROM alpine\nWORKDIR /src\nCOPY . .\nRUN ls -l \u0026\u0026 stop\n" + } + } +} +``` + +```console +$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test" "https://github.com/docker/cli.git#v20.10.11" +... + > [4/4] RUN ls -l && stop: +#8 0.136 drwxrwxrwx 5 root root 4096 Jul 27 18:31 kubernetes +#8 0.136 drwxrwxrwx 3 root root 4096 Jul 27 18:31 man +#8 0.136 drwxrwxrwx 2 root root 4096 Jul 27 18:31 opts +#8 0.136 -rw-rw-rw- 1 root root 1893 Jul 27 18:31 poule.yml +#8 0.136 drwxrwxrwx 7 root root 4096 Jul 27 18:31 scripts +#8 0.136 drwxrwxrwx 3 root root 4096 Jul 27 18:31 service +#8 0.136 drwxrwxrwx 2 root root 4096 Jul 27 18:31 templates +#8 0.136 drwxrwxrwx 10 root root 4096 Jul 27 18:31 vendor +#8 0.136 -rwxrwxrwx 1 root root 9620 Jul 27 18:31 vendor.conf +#8 0.136 /bin/sh: stop: not found +``` + +## Global scope attributes + +You can define global scope attributes in HCL/JSON and use them for code reuse +and setting values for variables. This means you can do a "data-only" HCL file +with the values you want to set/override and use it in the list of regular +output files. + +```hcl +# docker-bake.hcl +variable "FOO" { + default = "abc" +} + +target "app" { + args = { + v1 = "pre-${FOO}" + } +} +``` + +You can use this file directly: + +```console +$ docker buildx bake --print app +``` +```json +{ + "group": { + "default": { + "targets": [ + "app" + ] + } + }, + "target": { + "app": { + "context": ".", + "dockerfile": "Dockerfile", + "args": { + "v1": "pre-abc" + } + } + } +} +``` + +Or create an override configuration file: + +```hcl +# env.hcl +WHOAMI="myuser" +FOO="def-${WHOAMI}" +``` + +And invoke bake together with both of the files: + +```console +$ docker buildx bake -f docker-bake.hcl -f env.hcl --print app +``` +```json +{ + "group": { + "default": { + "targets": [ + "app" + ] + } + }, + "target": { + "app": { + "context": ".", + "dockerfile": "Dockerfile", + "args": { + "v1": "pre-def-myuser" + } + } + } +} +``` diff --git a/docs/guides/bake/hcl-vars-funcs.md b/docs/guides/bake/hcl-vars-funcs.md new file mode 100644 index 00000000..3f7a76f2 --- /dev/null +++ b/docs/guides/bake/hcl-vars-funcs.md @@ -0,0 +1,406 @@ +--- +title: "HCL variables and functions" +keywords: build, buildx, bake, buildkit, hcl +--- + +Similar to how Terraform provides a way to [define variables](https://www.terraform.io/docs/configuration/variables.html#declaring-an-input-variable), +the HCL file format also supports variable block definitions. These can be used +to define variables with values provided by the current environment, or a +default value when unset. + +A [set of generally useful functions](https://github.com/docker/buildx/blob/master/bake/hclparser/stdlib.go) +provided by [go-cty](https://github.com/zclconf/go-cty/tree/main/cty/function/stdlib) +are available for use in HCL files. In addition, [user defined functions](https://github.com/hashicorp/hcl/tree/main/ext/userfunc) +are also supported. + +## Using interpolation to tag an image with the git sha + +Bake supports variable blocks which are assigned to matching environment +variables or default values. + +```hcl +# docker-bake.hcl +variable "TAG" { + default = "latest" +} + +group "default" { + targets = ["webapp"] +} + +target "webapp" { + tags = ["docker.io/username/webapp:${TAG}"] +} +``` + +alternatively, in json format: + +```json +{ + "variable": { + "TAG": { + "default": "latest" + } + }, + "group": { + "default": { + "targets": ["webapp"] + } + }, + "target": { + "webapp": { + "tags": ["docker.io/username/webapp:${TAG}"] + } + } +} +``` + +```console +$ docker buildx bake --print webapp +``` +```json +{ + "group": { + "default": { + "targets": [ + "webapp" + ] + } + }, + "target": { + "webapp": { + "context": ".", + "dockerfile": "Dockerfile", + "tags": [ + "docker.io/username/webapp:latest" + ] + } + } +} +``` + +```console +$ TAG=$(git rev-parse --short HEAD) docker buildx bake --print webapp +``` +```json +{ + "group": { + "default": { + "targets": [ + "webapp" + ] + } + }, + "target": { + "webapp": { + "context": ".", + "dockerfile": "Dockerfile", + "tags": [ + "docker.io/username/webapp:985e9e9" + ] + } + } +} +``` + +## Using the `add` function + +You can use [`go-cty` stdlib functions](https://github.com/zclconf/go-cty/tree/main/cty/function/stdlib). +Here we are using the `add` function. + +```hcl +# docker-bake.hcl +variable "TAG" { + default = "latest" +} + +group "default" { + targets = ["webapp"] +} + +target "webapp" { + args = { + buildno = "${add(123, 1)}" + } +} +``` + +```console +$ docker buildx bake --print webapp +``` +```json +{ + "group": { + "default": { + "targets": [ + "webapp" + ] + } + }, + "target": { + "webapp": { + "context": ".", + "dockerfile": "Dockerfile", + "args": { + "buildno": "124" + } + } + } +} +``` + +## Defining an `increment` function + +It also supports [user defined functions](https://github.com/hashicorp/hcl/tree/main/ext/userfunc). +The following example defines a simple an `increment` function. + +```hcl +# docker-bake.hcl +function "increment" { + params = [number] + result = number + 1 +} + +group "default" { + targets = ["webapp"] +} + +target "webapp" { + args = { + buildno = "${increment(123)}" + } +} +``` + +```console +$ docker buildx bake --print webapp +``` +```json +{ + "group": { + "default": { + "targets": [ + "webapp" + ] + } + }, + "target": { + "webapp": { + "context": ".", + "dockerfile": "Dockerfile", + "args": { + "buildno": "124" + } + } + } +} +``` + +## Only adding tags if a variable is not empty using an `notequal` + +Here we are using the conditional `notequal` function which is just for +symmetry with the `equal` one. + +```hcl +# docker-bake.hcl +variable "TAG" {default="" } + +group "default" { + targets = [ + "webapp", + ] +} + +target "webapp" { + context="." + dockerfile="Dockerfile" + tags = [ + "my-image:latest", + notequal("",TAG) ? "my-image:${TAG}": "", + ] +} +``` + +```console +$ docker buildx bake --print webapp +``` +```json +{ + "group": { + "default": { + "targets": [ + "webapp" + ] + } + }, + "target": { + "webapp": { + "context": ".", + "dockerfile": "Dockerfile", + "tags": [ + "my-image:latest" + ] + } + } +} +``` + +## Using variables in functions + +You can refer variables to other variables like the target blocks can. Stdlib +functions can also be called but user functions can't at the moment. + +```hcl +# docker-bake.hcl +variable "REPO" { + default = "user/repo" +} + +function "tag" { + params = [tag] + result = ["${REPO}:${tag}"] +} + +target "webapp" { + tags = tag("v1") +} +``` + +```console +$ docker buildx bake --print webapp +``` +```json +{ + "group": { + "default": { + "targets": [ + "webapp" + ] + } + }, + "target": { + "webapp": { + "context": ".", + "dockerfile": "Dockerfile", + "tags": [ + "user/repo:v1" + ] + } + } +} +``` + +## Using variables in variables across files + +When multiple files are specified, one file can use variables defined in +another file. + +```hcl +# docker-bake1.hcl +variable "FOO" { + default = upper("${BASE}def") +} + +variable "BAR" { + default = "-${FOO}-" +} + +target "app" { + args = { + v1 = "pre-${BAR}" + } +} +``` + +```hcl +# docker-bake2.hcl +variable "BASE" { + default = "abc" +} + +target "app" { + args = { + v2 = "${FOO}-post" + } +} +``` + +```console +$ docker buildx bake -f docker-bake1.hcl -f docker-bake2.hcl --print app +``` +```json +{ + "group": { + "default": { + "targets": [ + "app" + ] + } + }, + "target": { + "app": { + "context": ".", + "dockerfile": "Dockerfile", + "args": { + "v1": "pre--ABCDEF-", + "v2": "ABCDEF-post" + } + } + } +} +``` + +## Using typed variables + +Non-string variables are also accepted. The value passed with env is parsed +into suitable type first. + +```hcl +# docker-bake.hcl +variable "FOO" { + default = 3 +} + +variable "IS_FOO" { + default = true +} + +target "app" { + args = { + v1 = FOO > 5 ? "higher" : "lower" + v2 = IS_FOO ? "yes" : "no" + } +} +``` + +```console +$ docker buildx bake --print app +``` +```json +{ + "group": { + "default": { + "targets": [ + "app" + ] + } + }, + "target": { + "app": { + "context": ".", + "dockerfile": "Dockerfile", + "args": { + "v1": "lower", + "v2": "yes" + } + } + } +} +``` + +## Built-in variables + +* `BAKE_CMD_CONTEXT` can be used to access the main `context` for bake command + from a bake file that has been [imported remotely](file-definition.md#remote-definition). +* `BAKE_LOCAL_PLATFORM` returns the current platform's default platform + specification (e.g. `linux/amd64`). diff --git a/docs/guides/bake/index.md b/docs/guides/bake/index.md new file mode 100644 index 00000000..88c5fecb --- /dev/null +++ b/docs/guides/bake/index.md @@ -0,0 +1,38 @@ +--- +title: "High-level build options with Bake" +keywords: build, buildx, bake, buildkit, hcl, json, compose +--- + +> This command is experimental. +> +> The design of bake is in early stages, and we are looking for [feedback from users](https://github.com/docker/buildx/issues). +{: .important } + +Buildx also aims to provide support for high-level build concepts that go beyond +invoking a single build command. We want to support building all the images in +your application together and let the users define project specific reusable +build flows that can then be easily invoked by anyone. + +[BuildKit](https://github.com/moby/buildkit) efficiently handles multiple +concurrent build requests and de-duplicating work. The build commands can be +combined with general-purpose command runners (for example, `make`). However, +these tools generally invoke builds in sequence and therefore cannot leverage +the full potential of BuildKit parallelization, or combine BuildKit's output +for the user. For this use case, we have added a command called +[`docker buildx bake`](https://docs.docker.com/engine/reference/commandline/buildx_bake/). + +The `bake` command supports building images from HCL, JSON and Compose files. +This is similar to [`docker compose build`](https://docs.docker.com/compose/reference/build/), +but allowing all the services to be built concurrently as part of a single +request. If multiple files are specified they are all read and configurations are +combined. + +We recommend using HCL files as its experience is more aligned with buildx UX +and also allows better code reuse, different target groups and extended features. + +## Next steps + +* [File definition](file-definition.md) +* [HCL variables and functions](hcl-vars-funcs.md) +* [Defining additional build contexts and linking targets](build-contexts.md) +* [Extension field with Compose](compose-xbake.md) diff --git a/docs/guides/registry-mirror.md b/docs/guides/registry-mirror.md index d84e2094..0218a9fb 100644 --- a/docs/guides/registry-mirror.md +++ b/docs/guides/registry-mirror.md @@ -14,7 +14,7 @@ debug = true mirrors = ["mirror.gcr.io"] ``` -> **Notes** +> **Note** > > `debug = true` has been added to be able to debug requests > in the BuildKit daemon and see if the mirror is effectively used. diff --git a/docs/reference/buildx_bake.md b/docs/reference/buildx_bake.md index 938754ab..4ad4aabb 100644 --- a/docs/reference/buildx_bake.md +++ b/docs/reference/buildx_bake.md @@ -34,12 +34,14 @@ Build from a file Bake is a high-level build command. Each specified target will run in parallel as part of the build. -Read [High-level build options](https://github.com/docker/buildx#high-level-build-options) -for introduction. +Read [High-level build options with Bake](https://docs.docker.com/build/bake/) +guide for introduction to writing bake files. -Please note that `buildx bake` command may receive backwards incompatible -features in the future if needed. We are looking for feedback on improving the -command and extending the functionality further. +> **Note** +> +> `buildx bake` command may receive backwards incompatible features in the future +> if needed. We are looking for feedback on improving the command and extending +> the functionality further. ## Examples @@ -49,167 +51,43 @@ Same as [`buildx --builder`](buildx.md#builder). ### Specify a build definition file (-f, --file) -By default, `buildx bake` looks for build definition files in the current -directory, the following are parsed: - -- `docker-compose.yml` -- `docker-compose.yaml` -- `docker-bake.json` -- `docker-bake.override.json` -- `docker-bake.hcl` -- `docker-bake.override.hcl` - -Use the `-f` / `--file` option to specify the build definition file to use. The -file can be a Docker Compose, JSON or HCL file. If multiple files are specified +Use the `-f` / `--file` option to specify the build definition file to use. +The file can be an HCL, JSON or Compose file. If multiple files are specified they are all read and configurations are combined. -The following example uses a Docker Compose file named `docker-compose.dev.yaml` -as build definition file, and builds all targets in the file: +You can pass the names of the targets to build, to build only specific target(s). +The following example builds the `db` and `webapp-release` targets that are +defined in the `docker-bake.dev.hcl` file: -```console -$ docker buildx bake -f docker-compose.dev.yaml - -[+] Building 66.3s (30/30) FINISHED - => [frontend internal] load build definition from Dockerfile 0.1s - => => transferring dockerfile: 36B 0.0s - => [backend internal] load build definition from Dockerfile 0.2s - => => transferring dockerfile: 3.73kB 0.0s - => [database internal] load build definition from Dockerfile 0.1s - => => transferring dockerfile: 5.77kB 0.0s - ... -``` - -Pass the names of the targets to build, to build only specific target(s). The -following example builds the `backend` and `database` targets that are defined -in the `docker-compose.dev.yaml` file, skipping the build for the `frontend` -target: - -```console -$ docker buildx bake -f docker-compose.dev.yaml backend database - -[+] Building 2.4s (13/13) FINISHED - => [backend internal] load build definition from Dockerfile 0.1s - => => transferring dockerfile: 81B 0.0s - => [database internal] load build definition from Dockerfile 0.2s - => => transferring dockerfile: 36B 0.0s - => [backend internal] load .dockerignore 0.3s - ... -``` - -You can also use a remote `git` bake definition: - -```console -$ docker buildx bake "https://github.com/docker/cli.git#v20.10.11" --print -#1 [internal] load git source https://github.com/docker/cli.git#v20.10.11 -#1 0.745 e8f1871b077b64bcb4a13334b7146492773769f7 refs/tags/v20.10.11 -#1 2.022 From https://github.com/docker/cli -#1 2.022 * [new tag] v20.10.11 -> v20.10.11 -#1 DONE 2.9s -{ - "group": { - "default": { - "targets": [ - "binary" - ] - } - }, - "target": { - "binary": { - "context": "https://github.com/docker/cli.git#v20.10.11", - "dockerfile": "Dockerfile", - "args": { - "BASE_VARIANT": "alpine", - "GO_STRIP": "", - "VERSION": "" - }, - "target": "binary", - "platforms": [ - "local" - ], - "output": [ - "build" - ] - } - } +```hcl +# docker-bake.dev.hcl +group "default" { + targets = ["db", "webapp-dev"] } -``` -As you can see the context is fixed to `https://github.com/docker/cli.git` even if -[no context is actually defined](https://github.com/docker/cli/blob/2776a6d694f988c0c1df61cad4bfac0f54e481c8/docker-bake.hcl#L17-L26) -in the definition. - -If you want to access the main context for bake command from a bake file -that has been imported remotely, you can use the `BAKE_CMD_CONTEXT` builtin var: - -```console -$ cat https://raw.githubusercontent.com/tonistiigi/buildx/remote-test/docker-bake.hcl -target "default" { - context = BAKE_CMD_CONTEXT - dockerfile-inline = < [4/4] RUN ls -l && stop: -#8 0.101 total 0 -#8 0.102 -rw-r--r-- 1 root root 0 Jul 27 18:47 bar -#8 0.102 -rw-r--r-- 1 root root 0 Jul 27 18:47 foo -#8 0.102 /bin/sh: stop: not found -``` -```console -$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test" "https://github.com/docker/cli.git#v20.10.11" --print -#1 [internal] load git source https://github.com/tonistiigi/buildx.git#remote-test -#1 0.429 577303add004dd7efeb13434d69ea030d35f7888 refs/heads/remote-test -#1 CACHED -{ - "target": { - "default": { - "context": "https://github.com/docker/cli.git#v20.10.11", - "dockerfile": "Dockerfile", - "dockerfile-inline": "FROM alpine\nWORKDIR /src\nCOPY . .\nRUN ls -l \u0026\u0026 stop\n" - } - } +target "db" { + dockerfile = "Dockerfile.db" + tags = ["docker.io/username/db"] } ``` ```console -$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test" "https://github.com/docker/cli.git#v20.10.11" -... - > [4/4] RUN ls -l && stop: -#8 0.136 drwxrwxrwx 5 root root 4096 Jul 27 18:31 kubernetes -#8 0.136 drwxrwxrwx 3 root root 4096 Jul 27 18:31 man -#8 0.136 drwxrwxrwx 2 root root 4096 Jul 27 18:31 opts -#8 0.136 -rw-rw-rw- 1 root root 1893 Jul 27 18:31 poule.yml -#8 0.136 drwxrwxrwx 7 root root 4096 Jul 27 18:31 scripts -#8 0.136 drwxrwxrwx 3 root root 4096 Jul 27 18:31 service -#8 0.136 drwxrwxrwx 2 root root 4096 Jul 27 18:31 templates -#8 0.136 drwxrwxrwx 10 root root 4096 Jul 27 18:31 vendor -#8 0.136 -rwxrwxrwx 1 root root 9620 Jul 27 18:31 vendor.conf -#8 0.136 /bin/sh: stop: not found +$ docker buildx bake -f docker-bake.dev.hcl db webapp-release ``` +See our [file definition](https://docs.docker.com/build/bake/file-definition/) +guide for more details. + ### Do not use cache when building the image (--no-cache) Same as `build --no-cache`. Do not use cache when building the image. @@ -286,701 +164,18 @@ $ docker buildx bake --set foo*.no-cache # bypass caching only for ``` Complete list of overridable fields: -`args`, `cache-from`, `cache-to`, `context`, `dockerfile`, `labels`, `no-cache`, -`output`, `platform`, `pull`, `secrets`, `ssh`, `tags`, `target` - -### File definition - -In addition to compose files, bake supports a JSON and an equivalent HCL file -format for defining build groups and targets. - -A target reflects a single docker build invocation with the same options that -you would specify for `docker build`. A group is a grouping of targets. - -Multiple files can include the same target and final build options will be -determined by merging them together. - -In the case of compose files, each service corresponds to a target. - -A group can specify its list of targets with the `targets` option. A target can -inherit build options by setting the `inherits` option to the list of targets or -groups to inherit from. - -Note: Design of bake command is work in progress, the user experience may change -based on feedback. - -HCL definition example: - -```hcl -group "default" { - targets = ["db", "webapp-dev"] -} - -target "webapp-dev" { - dockerfile = "Dockerfile.webapp" - tags = ["docker.io/username/webapp"] -} - -target "webapp-release" { - inherits = ["webapp-dev"] - platforms = ["linux/amd64", "linux/arm64"] -} - -target "db" { - dockerfile = "Dockerfile.db" - tags = ["docker.io/username/db"] -} -``` - -Complete list of valid target fields: - -`args`, `cache-from`, `cache-to`, `context`, `contexts`, `dockerfile`, `inherits`, `labels`, -`no-cache`, `no-cache-filter`, `output`, `platform`, `pull`, `secrets`, `ssh`, `tags`, `target` - -### Global scope attributes - -You can define global scope attributes in HCL/JSON and use them for code reuse -and setting values for variables. This means you can do a "data-only" HCL file -with the values you want to set/override and use it in the list of regular -output files. - -```hcl -# docker-bake.hcl -variable "FOO" { - default = "abc" -} - -target "app" { - args = { - v1 = "pre-${FOO}" - } -} -``` - -You can use this file directly: - -```console -$ docker buildx bake --print app -{ - "group": { - "default": { - "targets": [ - "app" - ] - } - }, - "target": { - "app": { - "context": ".", - "dockerfile": "Dockerfile", - "args": { - "v1": "pre-abc" - } - } - } -} -``` - -Or create an override configuration file: - -```hcl -# env.hcl -WHOAMI="myuser" -FOO="def-${WHOAMI}" -``` - -And invoke bake together with both of the files: - -```console -$ docker buildx bake -f docker-bake.hcl -f env.hcl --print app -{ - "group": { - "default": { - "targets": [ - "app" - ] - } - }, - "target": { - "app": { - "context": ".", - "dockerfile": "Dockerfile", - "args": { - "v1": "pre-def-myuser" - } - } - } -} -``` - -### HCL variables and functions - -Similar to how Terraform provides a way to [define variables](https://www.terraform.io/docs/configuration/variables.html#declaring-an-input-variable), -the HCL file format also supports variable block definitions. These can be used -to define variables with values provided by the current environment, or a -default value when unset. - -A [set of generally useful functions](https://github.com/docker/buildx/blob/master/bake/hclparser/stdlib.go) -provided by [go-cty](https://github.com/zclconf/go-cty/tree/main/cty/function/stdlib) -are available for use in HCL files. In addition, [user defined functions](https://github.com/hashicorp/hcl/tree/main/ext/userfunc) -are also supported. - -#### Using interpolation to tag an image with the git sha - -Bake supports variable blocks which are assigned to matching environment -variables or default values. - -```hcl -# docker-bake.hcl -variable "TAG" { - default = "latest" -} - -group "default" { - targets = ["webapp"] -} - -target "webapp" { - tags = ["docker.io/username/webapp:${TAG}"] -} -``` - -alternatively, in json format: - -```json -{ - "variable": { - "TAG": { - "default": "latest" - } - } - "group": { - "default": { - "targets": ["webapp"] - } - }, - "target": { - "webapp": { - "tags": ["docker.io/username/webapp:${TAG}"] - } - } -} -``` - -```console -$ docker buildx bake --print webapp -{ - "group": { - "default": { - "targets": [ - "webapp" - ] - } - }, - "target": { - "webapp": { - "context": ".", - "dockerfile": "Dockerfile", - "tags": [ - "docker.io/username/webapp:latest" - ] - } - } -} -``` - -```console -$ TAG=$(git rev-parse --short HEAD) docker buildx bake --print webapp -{ - "group": { - "default": { - "targets": [ - "webapp" - ] - } - }, - "target": { - "webapp": { - "context": ".", - "dockerfile": "Dockerfile", - "tags": [ - "docker.io/username/webapp:985e9e9" - ] - } - } -} -``` - -#### Using the `add` function - -You can use [`go-cty` stdlib functions](https://github.com/zclconf/go-cty/tree/main/cty/function/stdlib). -Here we are using the `add` function. - -```hcl -# docker-bake.hcl -variable "TAG" { - default = "latest" -} - -group "default" { - targets = ["webapp"] -} - -target "webapp" { - args = { - buildno = "${add(123, 1)}" - } -} -``` - -```console -$ docker buildx bake --print webapp -{ - "group": { - "default": { - "targets": [ - "webapp" - ] - } - }, - "target": { - "webapp": { - "context": ".", - "dockerfile": "Dockerfile", - "args": { - "buildno": "124" - } - } - } -} -``` - -#### Defining an `increment` function - -It also supports [user defined functions](https://github.com/hashicorp/hcl/tree/main/ext/userfunc). -The following example defines a simple an `increment` function. - -```hcl -# docker-bake.hcl -function "increment" { - params = [number] - result = number + 1 -} - -group "default" { - targets = ["webapp"] -} - -target "webapp" { - args = { - buildno = "${increment(123)}" - } -} -``` - -```console -$ docker buildx bake --print webapp -{ - "group": { - "default": { - "targets": [ - "webapp" - ] - } - }, - "target": { - "webapp": { - "context": ".", - "dockerfile": "Dockerfile", - "args": { - "buildno": "124" - } - } - } -} -``` - -#### Only adding tags if a variable is not empty using an `notequal` - -Here we are using the conditional `notequal` function which is just for -symmetry with the `equal` one. - -```hcl -# docker-bake.hcl -variable "TAG" {default="" } - -group "default" { - targets = [ - "webapp", - ] -} - -target "webapp" { - context="." - dockerfile="Dockerfile" - tags = [ - "my-image:latest", - notequal("",TAG) ? "my-image:${TAG}": "", - ] -} -``` - -```console -$ docker buildx bake --print webapp -{ - "group": { - "default": { - "targets": [ - "webapp" - ] - } - }, - "target": { - "webapp": { - "context": ".", - "dockerfile": "Dockerfile", - "tags": [ - "my-image:latest" - ] - } - } -} -``` - -#### Using variables in functions - -You can refer variables to other variables like the target blocks can. Stdlib -functions can also be called but user functions can't at the moment. - -```hcl -# docker-bake.hcl -variable "REPO" { - default = "user/repo" -} - -function "tag" { - params = [tag] - result = ["${REPO}:${tag}"] -} - -target "webapp" { - tags = tag("v1") -} -``` - -```console -$ docker buildx bake --print webapp -{ - "group": { - "default": { - "targets": [ - "webapp" - ] - } - }, - "target": { - "webapp": { - "context": ".", - "dockerfile": "Dockerfile", - "tags": [ - "user/repo:v1" - ] - } - } -} -``` - -#### Using variables in variables across files - -When multiple files are specified, one file can use variables defined in -another file. - -```hcl -# docker-bake1.hcl -variable "FOO" { - default = upper("${BASE}def") -} - -variable "BAR" { - default = "-${FOO}-" -} - -target "app" { - args = { - v1 = "pre-${BAR}" - } -} -``` - -```hcl -# docker-bake2.hcl -variable "BASE" { - default = "abc" -} - -target "app" { - args = { - v2 = "${FOO}-post" - } -} -``` - -```console -$ docker buildx bake -f docker-bake1.hcl -f docker-bake2.hcl --print app -{ - "group": { - "default": { - "targets": [ - "app" - ] - } - }, - "target": { - "app": { - "context": ".", - "dockerfile": "Dockerfile", - "args": { - "v1": "pre--ABCDEF-", - "v2": "ABCDEF-post" - } - } - } -} -``` - -#### Using typed variables - -Non-string variables are also accepted. The value passed with env is parsed -into suitable type first. - -```hcl -# docker-bake.hcl -variable "FOO" { - default = 3 -} - -variable "IS_FOO" { - default = true -} - -target "app" { - args = { - v1 = FOO > 5 ? "higher" : "lower" - v2 = IS_FOO ? "yes" : "no" - } -} -``` - -```console -$ docker buildx bake --print app -{ - "group": { - "default": { - "targets": [ - "app" - ] - } - }, - "target": { - "app": { - "context": ".", - "dockerfile": "Dockerfile", - "args": { - "v1": "lower", - "v2": "yes" - } - } - } -} -``` - -### Defining additional build contexts and linking targets - -In addition to the main `context` key that defines the build context each target can also define additional named contexts with a map defined with key `contexts`. These values map to the `--build-context` flag in the [build command](buildx_build.md#build-context). - -Inside the Dockerfile these contexts can be used with the `FROM` instruction or `--from` flag. - -The value can be a local source directory, container image (with docker-image:// prefix), Git URL, HTTP URL or a name of another target in the Bake file (with target: prefix). - -#### Pinning alpine image - -```Dockerfile -# Dockerfile -FROM alpine -RUN echo "Hello world" -``` - -```hcl -# docker-bake.hcl -target "app" { - contexts = { - alpine = "docker-image://alpine:3.13" - } -} -``` - -#### Using a secondary source directory - -```Dockerfile -# Dockerfile - -FROM scratch AS src - -FROM golang -COPY --from=src . . -``` - -```hcl -# docker-bake.hcl -target "app" { - contexts = { - src = "../path/to/source" - } -} -``` - -#### Using a result of one target as a base image in another target - -To use a result of one target as a build context of another, specity the target name with `target:` prefix. - -```Dockerfile -# Dockerfile -FROM baseapp -RUN echo "Hello world" -``` - -```hcl -# docker-bake.hcl - -target "base" { - dockerfile = "baseapp.Dockerfile" -} - -target "app" { - contexts = { - baseapp = "target:base" - } -} -``` - -Please note that in most cases you should just use a single multi-stage Dockerfile with multiple targets for similar behavior. This case is recommended when you have multiple Dockerfiles that can't be easily merged into one. - -### Extension field with Compose - -[Special extension](https://github.com/compose-spec/compose-spec/blob/master/spec.md#extension) -field `x-bake` can be used in your compose file to evaluate fields that are not -(yet) available in the [build definition](https://github.com/compose-spec/compose-spec/blob/master/build.md#build-definition). - -```yaml -# docker-compose.yml -services: - addon: - image: ct-addon:bar - build: - context: . - dockerfile: ./Dockerfile - args: - CT_ECR: foo - CT_TAG: bar - x-bake: - tags: - - ct-addon:foo - - ct-addon:alp - platforms: - - linux/amd64 - - linux/arm64 - cache-from: - - user/app:cache - - type=local,src=path/to/cache - cache-to: type=local,dest=path/to/cache - pull: true - - aws: - image: ct-fake-aws:bar - build: - dockerfile: ./aws.Dockerfile - args: - CT_ECR: foo - CT_TAG: bar - x-bake: - secret: - - id=mysecret,src=./secret - - id=mysecret2,src=./secret2 - platforms: linux/arm64 - output: type=docker - no-cache: true -``` - -```console -$ docker buildx bake --print -{ - "group": { - "default": { - "targets": [ - "aws", - "addon" - ] - } - }, - "target": { - "addon": { - "context": ".", - "dockerfile": "./Dockerfile", - "args": { - "CT_ECR": "foo", - "CT_TAG": "bar" - }, - "tags": [ - "ct-addon:foo", - "ct-addon:alp" - ], - "cache-from": [ - "user/app:cache", - "type=local,src=path/to/cache" - ], - "cache-to": [ - "type=local,dest=path/to/cache" - ], - "platforms": [ - "linux/amd64", - "linux/arm64" - ], - "pull": true - }, - "aws": { - "context": ".", - "dockerfile": "./aws.Dockerfile", - "args": { - "CT_ECR": "foo", - "CT_TAG": "bar" - }, - "tags": [ - "ct-fake-aws:bar" - ], - "secret": [ - "id=mysecret,src=./secret", - "id=mysecret2,src=./secret2" - ], - "platforms": [ - "linux/arm64" - ], - "output": [ - "type=docker" - ], - "no-cache": true - } - } -} -``` - -Complete list of valid fields for `x-bake`: - -`tags`, `cache-from`, `cache-to`, `secret`, `ssh`, `platforms`, `output`, -`pull`, `no-cache`, `no-cache-filter` - -### Built-in variables -* `BAKE_CMD_CONTEXT` can be used to access the main `context` for bake command -from a bake file that has been [imported remotely](#file). -* `BAKE_LOCAL_PLATFORM` returns the current platform's default platform -specification (e.g. `linux/amd64`). +* `args` +* `cache-from` +* `cache-to` +* `context` +* `dockerfile` +* `labels` +* `no-cache` +* `output` +* `platform` +* `pull` +* `secrets` +* `ssh` +* `tags` +* `target`