Commit Graph

124 Commits (master)

Author SHA1 Message Date
CrazyMax b0deb8bdd7
vendor: update docker/cli to f1615fa
also needs to update docker/docker to a60b458 (22.06 branch) otherwise
build breaks since docker/cli#3512 with:

    # github.com/docker/cli/cli/flags
    vendor/github.com/docker/cli/cli/flags/common.go:40:37: undefined: client.EnvOverrideCertPath
    vendor/github.com/docker/cli/cli/flags/common.go:41:37: undefined: client.EnvTLSVerify
    vendor/github.com/docker/cli/cli/flags/common.go:89:76: undefined: client.EnvOverrideHost

needs also to update github.com/spf13/cobra to v1.5.0 otherwise
build breaks with:

    # github.com/docker/cli/cli-plugins/plugin
    vendor/github.com/docker/cli/cli-plugins/plugin/plugin.go:130:4: unknown field 'HiddenDefaultCmd' in struct literal of type cobra.CompletionOptions

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
Justin Chadwell d9ef9bec34 kubernetes: add error when no pods available
This prevents the fall-through to the panic from division by zero in the
modulus below, and presents a neater error to the user.

Signed-off-by: Justin Chadwell <me@jedevc.com>
3 years ago
Justin Chadwell 1eff9310f8 remote: add additional connhelpers to buildx
Signed-off-by: Justin Chadwell <me@jedevc.com>
3 years ago
Tõnis Tiigi 062cf29de2
Merge pull request #1103 from jedevc/remote-driver-bootstrap
Use --bootstrap to wait for remote to become active
3 years ago
CrazyMax 57156ee95c
ls: adds fallback if buildkit version info unimplemented
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
Justin Chadwell c245f30a94 remote: use --bootstrap to wait for remote to become active
Signed-off-by: Justin Chadwell <me@jedevc.com>
3 years ago
Justin Chadwell d7e4affe98 Complete remote driver
This patch completes the work started in creating a remote driver:

- Renames the env driver to the remote driver (an alternative suggestion
  that should be more user-friendly)
- Adds support for TLS to encrypt connections with buildkitd
- Fixes outstanding review comments
- Reworks the buildx create command endpoint construction to be clearer
  and include better support for this new driver.

Signed-off-by: Justin Chadwell <me@jedevc.com>
3 years ago
Batuhan Apaydın 3dc83e5dd8 feat: env driver
Co-authored-by: Furkan Türkal <furkan.turkal@trendyol.com>
Signed-off-by: Batuhan Apaydın <batuhan.apaydin@trendyol.com>
3 years ago
Zsolt 4a226568a0 Fix tolerations not parsing its options correctly, add tests
Signed-off-by: Zsolt <zsolt.szeberenyi@figured.com>
3 years ago
Doug Borg 79de2c5d82 Only set default rootless image if it is not already customized
Only change the image to the default rootless image when using the
--rootless option if the image has not already customized with the
--image option.

Fix #938

Signed-off-by: Doug Borg <dougborg@apple.com>
3 years ago
Akihiro Suda add4301ed6
kubernetes: replace deprecated seccomp annotations with securityContext
Kubernetes added the official `securityContext.seccompProfile` support in Kubernetes 1.19.
Seccomp is still disabled by default.

The legacy `container.seccomp.security.alpha.kubernetes.io/<PODNAME>` annotation has been deprecated and will be unsupported in Kubernetes 1.25.
https://kubernetes.io/docs/tutorials/security/seccomp/

A test cluster can be created with the following minikube command:
```
minikube start --feature-gates SeccompDefault=true --extra-config kubelet.seccomp-default=true
```

Related to moby/buildkit PR 2782

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
3 years ago
Zsolt 3f6517747e Add support for defining kubernetes tolerations
Signed-off-by: Zsolt <zsolt.szeberenyi@figured.com>
3 years ago
Sebastiaan van Stijn fc7ba75fd7
Remove uses of deprecated io/ioutil
The package has been deprecated since Go 1.16: https://go.dev/doc/go1.16#ioutil

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
3 years ago
CrazyMax 971b5d2b73
vendor: update docker/cli to 8667ccd
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
Akihiro Suda 5f8600f098
docker-container: set UsernsMode only when needed
Set `UsernsMode="host"` only when the daemon is running in userns-remapping mode.

Fix issue 561

The issue will be also fixed in moby/moby PR 43084 (Docker 20.10.13).
This buildx PR helps users of old releases of Docker.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
3 years ago
Mayeul Blanzat 72dab552b5 Add an option to buildx rm to keep the buildkitd daemon running
Add --keep-daemon to the `rm` command option to preserve the buildkitd daemon after the buildx context is deleted.

Signed-off-by: Mayeul Blanzat <mayeul.blanzat@datadoghq.com>
3 years ago
Tonis Tiigi 40121c671c kubernetes: store config files for k8s
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
3 years ago
Tonis Tiigi 4c1621cccd store snapshot of config files on create
Files can be reused when container needs to be booted again.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
3 years ago
Tõnis Tiigi 49342dd54d
Merge pull request #787 from crazy-max/inject-certs
container driver: copy ca and user tls registries certs
3 years ago
CrazyMax 3f716f00fa
container driver: copy ca and user tls registries certs
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
Tonis Tiigi 2644d56a6d docker: dial session directly with http hijack
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
3 years ago
Tõnis Tiigi 461369748c
Merge pull request #782 from djs55/cgroup-parent
docker-container: place build containers in a separate cgroup
3 years ago
David Scott d5908cdddf docker-container: use /docker/buildx cgroup by default
This allows resource limits to be applied to all builds on a host.
For example to limit the total amount of CPU used by builds:

https://medium.com/@asishrs/docker-limit-resource-utilization-using-cgroup-parent-72a646651f9d

Signed-off-by: David Scott <dave@recoil.org>
3 years ago
David Scott b5bc754bad docker-container: support --driver-opt cgroup-parent=...
This allows the parent cgroup to be customised, which allows resource
limits to be imposed on build containers separately from "user"
containers.

Signed-off-by: David Scott <dave@recoil.org>
3 years ago
CrazyMax 9e62c9f074
Fix --driver flag usage markdown output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
Sebastiaan van Stijn 93867d02f0
container-driver: fix volume destination for cache
The container-driver creates a Linux container (as there currently isn't a
Windows version of buildkitd). However, the defaults are platform specific.

Buildx was using the defaults from the buildkit `util/appdefault' package,
which resulted in Buildx running on a Windows client to create a Linux
container that used the Windows location, which causes it to fail:

    invalid mount config for type "volume": invalid mount path: 'C:/ProgramData/buildkitd/.buildstate' mount path must be absolute

This patch hard-codes the destination to the default Linux path.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
3 years ago
Tonis Tiigi 422ba60b04 use long-running context for client initialization
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
3 years ago
Morlay c76b5eac03 feat(driver/kubernetes): support mount buildkit.toml and qemu installing
Signed-off-by: Morlay <morlay.null@gmail.com>
3 years ago
Tonis Tiigi 0b6ba1cd32 don't error on deleting old build containers without state volume
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
CrazyMax 258d12b2e7
Keep BuildKit state in a volume
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
4 years ago
Tonis Tiigi 9d88450118 enable opentelemetry support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Michal Augustyn a53e392afb feat: add resources handling to kubernetes driver
Signed-off-by: Michal Augustyn <michal.augustyn@mail.com>
4 years ago
Akihiro Suda 8b8725d1fd
set DefaultRootlessImage to "moby/buildkit:buildx-stable-1-rootless"
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
4 years ago
Tibor Vass 381dc8fb43 driver: do not insert "platform" as driver-opt
Addresses https://github.com/docker/setup-buildx-action/issues/45

Simple repro:
```
$ buildx create --platform linux/amd64 --use
$ buildx build - <<EOF
from scratch
EOF
```

Since https://github.com/docker/buildx/pull/370 a `platform` driver-opt was automatically inserted with the value specified by `--platform` flag on regardless of the type of driver, even though it was only used in the kubernetes driver. However, because the docker-container driver is pedantic about the options being passed, it errored out.

Another side-effect I suspect is that with the kubernetes driver it was now possible to specify the platforms in two different ways: `--driver-opt platform=...` and `--platform`.

This patch reverts completely the `platform` driver-opt and instead ensures the platforms information is passed onto the kubernetes driver via variables.

Signed-off-by: Tibor Vass <tibor@docker.com>
4 years ago
Tonis Tiigi f68f42cb11 build: add push support to docker driver
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tonis Tiigi 232af9aa0d move moby check to driver interface
Driver caching masked the method detection

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tonis Tiigi 1496ac9b55 util: simplify progress syncronization
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tonis Tiigi 290e25917c build: allow dockerfile from URL
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tõnis Tiigi 35a60b8e04
Merge pull request #441 from tonistiigi/buildkit-pull-creds2
refactor driver auth for easier passing
4 years ago
Andrew Haines 37020dc8da
Disable user namespace remapping in docker-container driver
Signed-off-by: Andrew Haines <andrew.haines@zencargo.com>
4 years ago
Tonis Tiigi 378bf70d4b refactor driver auth for easier passing
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tonis Tiigi 3b69482a2f docker-container: ensure credentials are passed when pulling buildkit image
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Sebastiaan van Stijn fa51b90094
vendor: fix docker/docker vendoring (update to 9f28837c1d93
commit c41b006be1 updated the version of
docker/docker in go.mod, but possibly overlooked that there was still a
replace rule present. As a result the version was not actually updated.

This patch removes the replace rule, updating docker/docker to 9f28837c1d93

full diff: 4634ce647c...9f28837c1d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
4 years ago
Tonis Tiigi c41b006be1 vendor: update buildkit to 2943a0838
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tonis Tiigi 2d720a1e0b vendor: update buildkit with typed errors support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Wang Jinglei a7c704c39d feat: enhance kubernetes driver
Signed-off-by: Wang Jinglei <morlay.null@gmail.com>
4 years ago
Wang Jinglei 844b901005 feat: use k8s cluster config when buildx used in k8s cluster
Signed-off-by: Wang Jinglei <morlay.null@gmail.com>
4 years ago
Tonis Tiigi 26570d05c1 docker-container: increase bootstrap timeout
Previous value was only 2 sec

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
5 years ago
Tonis Tiigi 8627f668f2 docker-container: show logs on bootstrap error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
5 years ago
Brian Goff 87fbc406f5 Make k8s driver priority lower
Otherwise it ends up being default and it's probably not the normal
case.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
5 years ago
l00397676 5b9d88b3ad docker-container: check local image store if pulling image failed
When booting `docker-container` driver, it will pull and run image
`moby/buildkit:buildx-stable-1`.
If current node cannot connect to dockerhub, driver `docker-container`
will always booting failure.
But user may already load the image manually or pull it from a priviate
registry.
Buildx should check local docker image store after pull failed.

Fixes: #199 issuecomment-561996661

Signed-off-by: Lu Jingxiao <lujingxiao@huawei.com>
5 years ago
Akihiro Suda c6f8de90aa kubernetes: show Kubernetes Pods as buildx "Nodes" in `docker buildx inspect`
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
5 years ago
Akihiro Suda 6b65b0c982 new driver: kubernetes
Tested with `kind` and GKE.

Note: "nodes" shown in `docker buildx ls` are unrelated to Kubernetes "nodes".
Probably buildx should come up with an alternative term.

Usage:

  $ kind create cluster
  $ export KUBECONFIG="$(kind get kubeconfig-path --name="kind")"

  $ docker buildx create --driver kubernetes --driver-opt replicas=3 --use
  $ docker buildx build -t foo --load .

`--load` loads the image into the local Docker.

Driver opts:

  - `image=IMAGE` - Sets the container image to be used for running buildkit.
  - `namespace=NS` - Sets the Kubernetes namespace. Defaults to the current namespace.
  - `replicas=N` - Sets the number of `Pod` replicas. Defaults to 1.
  - `rootless=(true|false)` - Run the container as a non-root user without `securityContext.privileged`. Defaults to false.
  - `loadbalance=(sticky|random)` - Load-balancing strategy. If set to "sticky", the pod is chosen using the hash of the context path. Defaults to "sticky"

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
5 years ago
Sune Keller fd44accc79 Support environment variables in docker-container driver
Fixes #169

Signed-off-by: Sune Keller <absukl@almbrand.dk>
5 years ago
Tonis Tiigi afd821010d docker-container: allow setting custom buildkit image
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi bcc882cbf1 docker-container: allow using host network
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi 75b80c277f driver: allow setting driver opts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi 8ed8795268 driver: allow setting buildkit config file
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Co-Authored-By: Tibor Vass <tiborvass@users.noreply.github.com>
6 years ago
Tonis Tiigi f0af89a204 driver: allow configuring buildkitd flags
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Akihiro Suda eee78c6c10 Put driver names to create --help
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
6 years ago
Tonis Tiigi 94a0422a1b rename to docker/buildx
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi e7eac2b909 build: use long running context for client
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi 5ddd3d2459 driver: remove unused version
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi cac37434aa build: add docker output for non-moby drivers
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi b7e15f3113 commands: implementation for inspect
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi bd3d5cd19e commands: add implementations for create, use, rm, stop
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi 0788035da8 driver: add feature testing
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi 68cad8e46b driver: add docker driver
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi 6469b05e33 driver: rename docker to docker-container
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi 15dae19cf8 driver: add client method
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi a6d893efca driver: add logging support to bootstrap
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi f302881c0d driver: start implemeting bootstrap for docker
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi 49f67b7e96 driver: docker driver base
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago
Tonis Tiigi 8438557ff7 driver: basic types startpoint
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
6 years ago