diff --git a/README.md b/README.md index a2fe299f..5a9af3ee 100644 --- a/README.md +++ b/README.md @@ -33,18 +33,25 @@ Key features: - [Working with builder instances](#working-with-builder-instances) - [Building multi-platform images](#building-multi-platform-images) - [High-level build options](#high-level-build-options) -- [Documentation](docs/reference/buildx.md) - - [`buildx bake`](docs/reference/buildx_bake.md) +- [Guides](docs/guides) + - [CI/CD](docs/guides/cicd.md) + - [CNI networking](docs/guides/cni-networking.md) + - [Registry mirror](docs/guides/registry-mirror.md) + - [Resource limiting](docs/guides/resource-limiting.md) + - [Using a custom network](docs/guides/custom-network.md) + - [Using a custom registry configuration](docs/guides/custom-registry-config.md) +- [Reference](docs/reference/buildx.md) + - [`buildx bake`](docs/reference/buildx_bake.md) - [`buildx build`](docs/reference/buildx_build.md) - [`buildx create`](docs/reference/buildx_create.md) - [`buildx du`](docs/reference/buildx_du.md) - [`buildx imagetools`](docs/reference/buildx_imagetools.md) - [`buildx imagetools create`](docs/reference/buildx_imagetools_create.md) - - [`buildx imagetools inspect`](docs/reference/buildx_imagetools_inspect.md) + - [`buildx imagetools inspect`](docs/reference/buildx_imagetools_inspect.md) - [`buildx inspect`](docs/reference/buildx_inspect.md) - [`buildx install`](docs/reference/buildx_install.md) - - [`buildx ls`](docs/reference/buildx_ls.md) - - [`buildx prune`](docs/reference/buildx_prune.md) + - [`buildx ls`](docs/reference/buildx_ls.md) + - [`buildx prune`](docs/reference/buildx_prune.md) - [`buildx rm`](docs/reference/buildx_rm.md) - [`buildx stop`](docs/reference/buildx_stop.md) - [`buildx uninstall`](docs/reference/buildx_uninstall.md) diff --git a/docs/guides/cicd.md b/docs/guides/cicd.md new file mode 100644 index 00000000..5119c091 --- /dev/null +++ b/docs/guides/cicd.md @@ -0,0 +1,48 @@ +# CI/CD + +## GitHub Actions + +Docker provides a [GitHub Action that will build and push your image](https://github.com/docker/build-push-action/#about) +using Buildx. Here is a simple workflow: + +```yaml +name: ci + +on: + push: + branches: + - 'main' + +jobs: + docker: + runs-on: ubuntu-latest + steps: + - + name: Set up QEMU + uses: docker/setup-qemu-action@v1 + - + name: Set up Docker Buildx + uses: docker/setup-buildx-action@v1 + - + name: Login to DockerHub + uses: docker/login-action@v1 + with: + username: ${{ secrets.DOCKERHUB_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + - + name: Build and push + uses: docker/build-push-action@v2 + with: + push: true + tags: user/app:latest +``` + +In this example we are also using 3 other actions: + +* [`setup-buildx`](https://github.com/docker/setup-buildx-action) action will create and boot a builder using by +default the `docker-container` [builder driver](../reference/buildx_create.md#driver). +This is **not required but recommended** using it to be able to build multi-platform images, export cache, etc. +* [`setup-qemu`](https://github.com/docker/setup-qemu-action) action can be useful if you want +to add emulation support with QEMU to be able to build against more platforms. +* [`login`](https://github.com/docker/login-action) action will take care to log +in against a Docker registry. diff --git a/docs/guides/cni-networking.md b/docs/guides/cni-networking.md new file mode 100644 index 00000000..07aa4cb4 --- /dev/null +++ b/docs/guides/cni-networking.md @@ -0,0 +1,23 @@ +# CNI networking + +It can be useful to use a bridge network for your builder if for example you +encounter a network port contention during multiple builds. If you're using +the BuildKit image, CNI is not yet available in it, but you can create +[a custom BuildKit image with CNI support](https://github.com/moby/buildkit/blob/master/docs/cni-networking.md). + +Now build this image: + +```console +$ docker buildx build --tag buildkit-cni:local --load . +``` + +Then [create a `docker-container` builder](../reference/buildx_create.md) that +will use this image: + +```console +$ docker buildx create --use \ + --name mybuilder \ + --driver docker-container \ + --driver-opt "image=buildkit-cni:local" \ + --buildkitd-flags "--oci-worker-net=cni" +``` diff --git a/docs/guides/custom-network.md b/docs/guides/custom-network.md new file mode 100644 index 00000000..4377f290 --- /dev/null +++ b/docs/guides/custom-network.md @@ -0,0 +1,48 @@ +# Using a custom network + +[Create a network](https://docs.docker.com/engine/reference/commandline/network_create/) +named `foonet`: + +```console +$ docker network create foonet +``` + +[Create a `docker-container` builder](../reference/buildx_create.md) named +`mybuilder` that will use this network: + +```console +$ docker buildx create --use \ + --name mybuilder \ + --driver docker-container \ + --driver-opt "network=foonet" +``` + +Boot and [inspect `mybuilder`](../reference/buildx_inspect.md): + +```console +$ docker buildx inspect --bootstrap +``` + +[Inspect the builder container](https://docs.docker.com/engine/reference/commandline/inspect/) +and see what network is being used: + +```console +$ docker inspect buildx_buildkit_mybuilder0 --format={{.NetworkSettings.Networks}} +map[foonet:0xc00018c0c0] +``` + +## What's `buildx_buildkit_mybuilder0`? + +`buildx_buildkit_mybuilder0` is the container name. It can be broken down like this: + +* `buildx_buildkit_` is a hardcoded prefix +* `mybuilder0` is the name of the node (defaults to builder name + position in the list of nodes) + +```console +$ docker buildx ls +NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS +mybuilder * docker-container + mybuilder0 unix:///var/run/docker.sock running linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6 +default docker + default default running linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6 +``` diff --git a/docs/guides/custom-registry-config.md b/docs/guides/custom-registry-config.md new file mode 100644 index 00000000..4d702b51 --- /dev/null +++ b/docs/guides/custom-registry-config.md @@ -0,0 +1,63 @@ +# Using a custom registry configuration + +If you [create a `docker-container` or `kubernetes` builder](../reference/buildx_create.md) and +have specified certificates for registries in the [BuildKit daemon configuration](https://github.com/moby/buildkit/blob/master/docs/buildkitd.toml.md), +the files will be copied into the container under `/etc/buildkit/certs` and +configuration will be updated to reflect that. + +Take the following `buildkitd.toml` configuration that will be used for +pushing an image to this registry using self-signed certificates: + +```toml" +debug = true +[registry."myregistry.com"] + ca=["/etc/certs/myregistry.pem"] + [[registry."myregistry.com".keypair]] + key="/etc/certs/myregistry_key.pem" + cert="/etc/certs/myregistry_cert.pem" +``` +> `/etc/buildkitd.toml` + +Here we have configured a self-signed certificate for `myregistry.com` registry. + +Now [create a `docker-container` builder](../reference/buildx_create.md) +that will use this BuildKit configuration: + +```console +$ docker buildx create --use \ + --name mybuilder \ + --driver docker-container \ + --config /etc/buildkitd.toml +``` + +Inspecting the builder container, you can see that buildkitd configuration +has changed: + +```console +$ docker exec -it buildx_buildkit_mybuilder0 cat /etc/buildkit/buildkitd.toml +``` +```toml +debug = true + +[registry] + + [registry."myregistry.com"] + ca = ["/etc/buildkit/certs/myregistry.com/myregistry.pem"] + + [[registry."myregistry.com".keypair]] + cert = "/etc/buildkit/certs/myregistry.com/myregistry_cert.pem" + key = "/etc/buildkit/certs/myregistry.com/myregistry_key.pem" +``` + +And certificates copied inside the container: + +```console +$ docker exec -it buildx_buildkit_mybuilder0 ls /etc/buildkit/certs/myregistry.com/ +myregistry.pem myregistry_cert.pem myregistry_key.pem +``` + +Now you should be able to push to the registry with this builder: + +```console +$ docker buildx build --push --tag myregistry.com/myimage:latest . +``` diff --git a/docs/guides/opentelemetry.md b/docs/guides/opentelemetry.md new file mode 100644 index 00000000..0d67750a --- /dev/null +++ b/docs/guides/opentelemetry.md @@ -0,0 +1,31 @@ +# OpenTelemetry support + +To capture the trace to [Jaeger](https://github.com/jaegertracing/jaeger), set +`JAEGER_TRACE` environment variable to the collection address using a `driver-opt`. + +First create a Jaeger container: + +```console +$ docker run -d --name jaeger -p "6831:6831/udp" -p "16686:16686" jaegertracing/all-in-one +``` + +Then [create a `docker-container` builder](../reference/buildx_create.md) +that will use the Jaeger instance via the `JAEGER_TRACE` env var: + +```console +$ docker buildx create --use \ + --name mybuilder \ + --driver docker-container \ + --driver-opt "network=host" \ + --driver-opt "env.JAEGER_TRACE=localhost:6831" +``` + +Boot and [inspect `mybuilder`](../reference/buildx_inspect.md): + +```console +$ docker buildx inspect --bootstrap +``` + +Buildx commands should be traced at `http://127.0.0.1:16686/`: + +![](https://user-images.githubusercontent.com/1951866/124468052-ef085400-dd98-11eb-84ab-7ac8e261dd52.png) diff --git a/docs/guides/registry-mirror.md b/docs/guides/registry-mirror.md new file mode 100644 index 00000000..22f36133 --- /dev/null +++ b/docs/guides/registry-mirror.md @@ -0,0 +1,60 @@ +# Registry mirror + +You can define a registry mirror to use for your builds by providing a [BuildKit daemon configuration](https://github.com/moby/buildkit/blob/master/docs/buildkitd.toml.md) +while creating a builder with the [`--config` flags](../reference/buildx_create.md#config). + +```toml +debug = true +[registry."docker.io"] + mirrors = ["mirror.gcr.io"] +``` +> `/etc/buildkitd.toml` + +> :information_source: `debug = true` has been added to be able to debug requests +in the BuildKit daemon and see if the mirror is effectively used. + +Then [create a `docker-container` builder](../reference/buildx_create.md) +that will use this BuildKit configuration: + +```console +$ docker buildx create --use \ + --name mybuilder \ + --driver docker-container \ + --config /etc/buildkitd.toml +``` + +Boot and [inspect `mybuilder`](../reference/buildx_inspect.md): + +```console +$ docker buildx inspect --bootstrap +``` + +Build an image: + +```console +$ docker buildx build --load . -f-< `/etc/buildkitd.toml` + +Now you can [create a `docker-container` builder](../reference/buildx_create.md) +that will use this BuildKit configuration to limit parallelism. + +```console +$ docker buildx create --use \ + --name mybuilder \ + --driver docker-container \ + --config /etc/buildkitd.toml +``` + +## Limit on TCP connections + +We are also now limiting TCP connections to **4 per registry** with an additional +connection not used for layer pulls and pushes. This limitation will be able to +manage TCP connection per host to avoid your build being stuck while pulling +images. The additional connection is used for metadata requests +(image config retrieval) to enhance the overall build time. + +More info: [moby/buildkit#2259](https://github.com/moby/buildkit/pull/2259)