From 68ef5b9c9b95b8b75bd5faaa90b0c98c9c28098a Mon Sep 17 00:00:00 2001 From: Justin Chadwell Date: Mon, 25 Jul 2022 13:15:49 +0100 Subject: [PATCH 1/3] docs: create dedicated drivers section Create a dedicated folder for information on drivers, and write a new index.md with content adapted from the README, and a new feature comparisons table. Signed-off-by: Justin Chadwell --- README.md | 34 ++++++--------- docs/guides/drivers/index.md | 41 +++++++++++++++++++ .../kubernetes.md} | 2 +- .../{remote-builder.md => drivers/remote.md} | 2 +- 4 files changed, 55 insertions(+), 24 deletions(-) create mode 100644 docs/guides/drivers/index.md rename docs/guides/{kubernetes-builder.md => drivers/kubernetes.md} (99%) rename docs/guides/{remote-builder.md => drivers/remote.md} (99%) diff --git a/README.md b/README.md index 0f8189f1..319c7689 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,7 @@ Key features: - [Using a custom registry configuration](docs/guides/custom-registry-config.md) - [OpenTelemetry support](docs/guides/opentelemetry.md) - [Registry mirror](docs/guides/registry-mirror.md) - - [Remote builder](docs/guides/remote-builder.md) + - [Drivers](docs/guides/drivers/index.md) - [Resource limiting](docs/guides/resource-limiting.md) - [Reference](docs/reference/buildx.md) - [`buildx bake`](docs/reference/buildx_bake.md) @@ -185,27 +185,17 @@ specifying target platform. In addition, Buildx also supports new features that are not yet available for regular `docker build` like building manifest lists, distributed caching, and exporting build results to OCI image tarballs. -Buildx is supposed to be flexible and can be run in different configurations -that are exposed through a driver concept. Currently, we support: - -- a [`docker` driver](docs/reference/buildx_create.md#docker-driver) that uses - the BuildKit library bundled into the Docker daemon binary, -- a [`docker-container` driver](docs/reference/buildx_create.md#docker-container-driver) - that automatically launches BuildKit inside a Docker container, -- a [`kubernetes` driver](docs/reference/buildx_create.md#kubernetes-driver) to - spin up pods with defined BuildKit container image to build your images. -- a [`remote` driver](docs/reference/buildx_create.md#remote-driver) to - connect to manually provisioned and managed buildkitd instances. - -We plan to add more drivers in the future. - -The user experience of using buildx is very similar across drivers, but there -are some features that are not currently supported by the `docker` driver, -because the BuildKit library bundled into docker daemon currently uses a -different storage component. In contrast, all images built with `docker` driver -are automatically added to the `docker images` view by default, whereas when -using other drivers the method for outputting an image needs to be selected -with `--output`. +Buildx is flexible and can be run in different configurations that are exposed +through various "drivers". Each driver defines how and where a build should +run, and have different feature sets. + +We currently support the following drivers: +- The `docker` driver ([reference](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver)) +- The `docker-container` driver ([guide](./docker-container.md), [reference](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver)) +- The `kubernetes` driver ([guide](./kubernetes.md), [reference](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver)) +- The `remote` driver ([guide](./remote.md)) + +For more information on drivers, see the [drivers guide](docs/guides/drivers/index.md). ## Working with builder instances diff --git a/docs/guides/drivers/index.md b/docs/guides/drivers/index.md new file mode 100644 index 00000000..898b6946 --- /dev/null +++ b/docs/guides/drivers/index.md @@ -0,0 +1,41 @@ +# Buildx drivers overview + +The buildx client connects out to the BuildKit backend to execute builds - +Buildx drivers allow fine-grained control over management of the backend, and +supports several different options for where and how BuildKit should run. + +Currently, we support the following drivers: + +- The `docker` driver, that uses the BuildKit library bundled into the Docker + daemon. + ([reference](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver)) +- The `docker-container` driver, that launches a dedicated BuildKit container + using Docker, for access to advanced features. + ([guide](./docker-container.md), [reference](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver)) +- The `kubernetes` driver, that launches dedicated BuildKit pods in a + remote Kubernetes cluster, for scalable builds. + ([guide](./kubernetes.md), [reference](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver)) +- The `remote` driver, that allows directly connecting to a manually managed + BuildKit daemon, for more custom setups. + ([guide](./remote.md)) + + + +To create a new builder that uses one of the above drivers, you can use the +[`docker buildx create`](https://docs.docker.com/engine/reference/commandline/buildx_create/) command: + +```console +$ docker buildx create --name= --driver= --driver-opt= +``` + +The build experience is very similar across drivers, however, there are some +features that are not evenly supported across the board, notably, the `docker` +driver does not include support for certain output/caching types. + +| Feature | `docker` | `docker-container` | `kubernetes` | `remote` | +| :---------------------------- | :-------------: | :----------------: | :----------: | :--------------------: | +| **Automatic `--load`** | ✅ | ❌ | ❌ | ❌ | +| **Cache export** | ❔ (inline only) | ✅ | ✅ | ✅ | +| **Docker/OCI tarball output** | ❌ | ✅ | ✅ | ✅ | +| **Multi-arch images** | ❌ | ✅ | ✅ | ✅ | +| **BuildKit configuration** | ❌ | ✅ | ✅ | ❔ (managed externally) | diff --git a/docs/guides/kubernetes-builder.md b/docs/guides/drivers/kubernetes.md similarity index 99% rename from docs/guides/kubernetes-builder.md rename to docs/guides/drivers/kubernetes.md index aa856be2..240256fb 100644 --- a/docs/guides/kubernetes-builder.md +++ b/docs/guides/drivers/kubernetes.md @@ -1,4 +1,4 @@ -# Kubernetes builder +# Kubernetes driver The buildx kubernetes driver allows connecting your local development or ci environments to your kubernetes cluster to allow access to more powerful diff --git a/docs/guides/remote-builder.md b/docs/guides/drivers/remote.md similarity index 99% rename from docs/guides/remote-builder.md rename to docs/guides/drivers/remote.md index 7caf5825..5e61245b 100644 --- a/docs/guides/remote-builder.md +++ b/docs/guides/drivers/remote.md @@ -1,4 +1,4 @@ -# Remote builder +# Remote driver The buildx remote driver allows for more complex custom build workloads that allow users to connect to external buildkit instances. This is useful for From f2b504b77d2c9939fecd8d6c2a42a11a5354f385 Mon Sep 17 00:00:00 2001 From: Justin Chadwell Date: Mon, 25 Jul 2022 13:40:23 +0100 Subject: [PATCH 2/3] docs: add basic docker-container driver guide Signed-off-by: Justin Chadwell --- docs/guides/drivers/docker-container.md | 69 +++++++++++++++++++++++++ 1 file changed, 69 insertions(+) create mode 100644 docs/guides/drivers/docker-container.md diff --git a/docs/guides/drivers/docker-container.md b/docs/guides/drivers/docker-container.md new file mode 100644 index 00000000..25c1ff0d --- /dev/null +++ b/docs/guides/drivers/docker-container.md @@ -0,0 +1,69 @@ +# Docker container driver + +The buildx docker-container driver allows creation of a managed and +customizable BuildKit environment inside a dedicated Docker container. + +Using the docker-container driver has a couple of advantages over the basic +docker driver. Firstly, we can manually override the version of buildkit to +use, meaning that we can access the latest and greatest features as soon as +they're released, instead of waiting to upgrade to a newer version of Docker. +Additionally, we can access more complex features like multi-architecture +builds and the more advanced cache exporters, which are currently unsupported +in the default docker driver. + +We can easily create a new builder that uses the docker-container driver: + +```console +$ docker buildx create --name container --driver docker-container +container +``` + +We should then be able to see it on our list of available builders: + +```console +$ docker buildx ls +NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS +container docker-container + container0 desktop-linux inactive +default docker + default default running 20.10.17 linux/amd64, linux/386 +``` + +If we trigger a build, the appropriate `moby/buildkit` image will be pulled +from [Docker Hub](https://hub.docker.com/u/moby/buildkit), the image started, +and our build submitted to our containerized build server. + +```console +$ docker buildx build -t --builder=container . +WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load +#1 [internal] booting buildkit +#1 pulling image moby/buildkit:buildx-stable-1 +#1 pulling image moby/buildkit:buildx-stable-1 1.9s done +#1 creating container buildx_buildkit_container0 +#1 creating container buildx_buildkit_container0 0.5s done +#1 DONE 2.4s +... +``` + +Note the warning "Build result will only remain in the build cache" - unlike +the `docker` driver, the built image must be explicitly loaded into the local +image store. We can use the `--load` flag for this: + +```console +$ docker buildx build --load -t --builder=container . +... + => exporting to oci image format 7.7s + => => exporting layers 4.9s + => => exporting manifest sha256:4e4ca161fa338be2c303445411900ebbc5fc086153a0b846ac12996960b479d3 0.0s + => => exporting config sha256:adf3eec768a14b6e183a1010cb96d91155a82fd722a1091440c88f3747f1f53f 0.0s + => => sending tarball 2.8s + => importing to docker +``` + +The image should then be available in the image store: + +```console +$ docker image ls +REPOSITORY TAG IMAGE ID CREATED SIZE + latest adf3eec768a1 2 minutes ago 197MB +``` From a42eb73043bdf464ac5698b76e9e27e0269d3dca Mon Sep 17 00:00:00 2001 From: Justin Chadwell Date: Mon, 25 Jul 2022 13:42:28 +0100 Subject: [PATCH 3/3] docs: add further reading section for drivers Signed-off-by: Justin Chadwell --- docs/guides/drivers/docker-container.md | 6 ++++++ docs/guides/drivers/kubernetes.md | 6 ++++++ docs/guides/drivers/remote.md | 2 ++ 3 files changed, 14 insertions(+) diff --git a/docs/guides/drivers/docker-container.md b/docs/guides/drivers/docker-container.md index 25c1ff0d..3b0dacb6 100644 --- a/docs/guides/drivers/docker-container.md +++ b/docs/guides/drivers/docker-container.md @@ -67,3 +67,9 @@ $ docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE latest adf3eec768a1 2 minutes ago 197MB ``` + +## Further reading + +For more information on the docker-container driver, see the [buildx reference](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver). + + diff --git a/docs/guides/drivers/kubernetes.md b/docs/guides/drivers/kubernetes.md index 240256fb..4b661931 100644 --- a/docs/guides/drivers/kubernetes.md +++ b/docs/guides/drivers/kubernetes.md @@ -230,3 +230,9 @@ $ docker buildx create \ ``` This will create your pods without `securityContext.privileged`. + +## Further reading + +For more information on the kubernetes driver, see the [buildx reference](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver). + + diff --git a/docs/guides/drivers/remote.md b/docs/guides/drivers/remote.md index 5e61245b..b0a314e3 100644 --- a/docs/guides/drivers/remote.md +++ b/docs/guides/drivers/remote.md @@ -174,3 +174,5 @@ $ docker buildx create \ --driver remote \ kube-pod://buildkitd-XXXXXXXXXX-xxxxx ``` + + \ No newline at end of file