Commit Graph

191 Commits (master)

Author SHA1 Message Date
CrazyMax fb61fde581
fix cannot set environment variables in parallel tests
> FAIL: TestReadTargets/ArgsOverrides/leaf (0.00s)
> panic: testing: t.Setenv called after t.Parallel; cannot set environment variables in parallel tests

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
Justin Chadwell fbb4f4dec8 bake: avoid nesting error diagnostics
With changes to the lazy evaluation, the evaluation order is no longer
fixed - this means that we can follow long and confusing paths to get to
an error.

Because of the co-recursive nature of the lazy evaluation, we need to
take special care that the original HCL diagnostics are not discarded
and are preserved so that the original source of the error can be
detected. Preserving the full trace is not necessary, and probably not
useful to the user - all of the file that is not lazily loaded will be
eagerly loaded after all struct blocks are loaded - so the error would
be found regardless.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell dc8a2b0398 bake: avoid early-exit for resolution failures
With changes made to allow lazy evaluation, we were early exiting if an
undefined name was detected, either for a variable or a function.

This had two key implications:

1. The error messages changed, and became significantly less
   informative.

   For example, we went from:

   > Unknown variable; There is no variable named "FO". Did you mean "FOO"?, and 1 other diagnostic(s)

   To

   > Invalid expression; undefined variable "FO"

2. Any issues in our function detection from funcCalls which cause JSON
   functions to be erroneously detected cause invalid functions to be
   resolved, which causes new name resolution errors.

To avoid the above problems, we can defer the error from an undefined
name until HCL evaluation - which produces the more informative errors,
and does not suffer from incorrectly detecting JSON functions.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
CrazyMax 21ac4c34fb
update github.com/compose-spec/compose-go to v1.9.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
Justin Chadwell 4bdf98cf20 lint: ban fmt.Errorf in preference of errors.Errorf
Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell 48357ee0c6 bake: fix loop references
Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Akihiro Suda 1f56f51740
bake: SOURCE_DATE_EPOCH: fix `panic: assignment to entry in nil map`
Fix issue 1562

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2 years ago
Akihiro Suda 0e6f5a155e
Propagate SOURCE_DATE_EPOCH from the client env
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2 years ago
CrazyMax fe76a1b179
bake: support null label value
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
CrazyMax df4957307f
bake: support null arg value
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
Justin Chadwell e51b55e03c
bake: add tests for block interpolation
Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell 296b8249cb
bake: support block-based interpolation
This patch adds support for block-based interpolation, so that
properties of blocks can be referenced in the current block and across
other blocks.

Previously, order-of-evaluation did not matter for blocks, and could be
evaluated in any order. However, now that blocks can refer to each
other, we split out this dynamic evaluation order into a separate
resolveBlock function.

Additionally, we need to support partial block evaluations - if block A
refers to property X of block B, when we should only evaluate property
X, and not the entire block. This ensures that we can safely evaluate
blocks that refer to other properties within themselves, and allows
sequences that would otherwise be co-recursive. We take special care in
this logic to ensure that each property is evaluated once *and only*
once - this could otherwise present inconsistencies with stateful
functions, and could risk inconsistent results.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell 7c6b840199
bake: add cty tags to hcl structures
Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
Justin Chadwell 25aa893bad bake: add attests field
Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
CrazyMax e7b5ee7518
mutualize builder logic
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
CrazyMax 5b4e8b9d71
hcl: SrcRange not checked when solving JSON func calls
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
CrazyMax 48b573e835
bake: fix panic for unsupported hcl variable type
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
CrazyMax e66410b932
Merge pull request #1313 from jedevc/bake-group-recurse
bake: recursively resolve groups
2 years ago
David Karlsson be3b41acc6 docs: refactored file and directory structure
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2 years ago
Justin Chadwell 77b33260f8 bake: recursively resolve groups
Groups that contained other groups were not recursively resolved by
ReadTargets, which prevented output from --print from being useable as a
self-contained bake file.

This patch ensures that all groups that are referenced inside the bake
file are actually defined under the groups field. This has required a
substantial refactor, as previously only a single group was returned
from ReadTargets, notably, returning a map of groups, instead of a
slice.

This does introduce a small behavior change to the behavior of --print -
while previously, passing a group name to bake would return all the
targets of that group back as the default group, now only the name of
that group will be inserted into the default group, keeping the original
group intact. The impact of this can be observed in some of the changes
to the bake_test.go file.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2 years ago
CrazyMax 9c22be5d9c
bake: test compose file validation
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
CrazyMax 42dea89247
bake: test for unknown extensions
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
CrazyMax 982a332679
bake(compose): fix unskipped services without build context
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
CrazyMax 7d8a6bc1d7
bake: load .env file from working dir for compose files
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
CrazyMax a378f8095e
test: misplaced expected value in assert
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
CrazyMax 7f572eb044
bake: contexts support with x-bake
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
Tõnis Tiigi 19641ec8ca
Merge pull request #1214 from jedevc/timestamp-func
bake: add timestamp function
2 years ago
CrazyMax 7ef679d945
bake(compose): allow dot in target name
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2 years ago
CrazyMax b0deb8bdd7
vendor: update docker/cli to f1615fa
also needs to update docker/docker to a60b458 (22.06 branch) otherwise
build breaks since docker/cli#3512 with:

    # github.com/docker/cli/cli/flags
    vendor/github.com/docker/cli/cli/flags/common.go:40:37: undefined: client.EnvOverrideCertPath
    vendor/github.com/docker/cli/cli/flags/common.go:41:37: undefined: client.EnvTLSVerify
    vendor/github.com/docker/cli/cli/flags/common.go:89:76: undefined: client.EnvOverrideHost

needs also to update github.com/spf13/cobra to v1.5.0 otherwise
build breaks with:

    # github.com/docker/cli/cli-plugins/plugin
    vendor/github.com/docker/cli/cli-plugins/plugin/plugin.go:130:4: unknown field 'HiddenDefaultCmd' in struct literal of type cobra.CompletionOptions

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
Tõnis Tiigi 11c1e03e93
Merge pull request #1155 from crazy-max/compose-cacheto
bake: support compose build cache_to
3 years ago
Justin Chadwell 691723f9f9 bake: add timestamp function
Terraform includes a timestamp function to get the current time. go-cty
has imported a number of the timestamp functions to it's standard
library, however, this was one was not included.

This patch simply pulls in the TimestampFunc from Terraform's
internal/lang/funcs/datetime.go to allow easily fetching the current
time in bake.

Signed-off-by: Justin Chadwell <me@jedevc.com>
3 years ago
CrazyMax 5b452b72a2
Merge pull request #1198 from jedevc/bump-buildkit-f4eb826
Bump buildkit to master branch
3 years ago
Justin Chadwell 27fcb73c7c bake: add tests for missing attributes in userfuncs
Signed-off-by: Justin Chadwell <me@jedevc.com>
3 years ago
Justin Chadwell 2aa22597f0 bake: forbid empty result and params in userfuncs
Signed-off-by: Justin Chadwell <me@jedevc.com>
3 years ago
Justin Chadwell 3b4780ef19 vendor: bump buildkit to master
Signed-off-by: Justin Chadwell <me@jedevc.com>
3 years ago
CrazyMax 12fde33d9b
bake: dedup compose main and extension fields values
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax a0f92829a7
bake: merge cache-from field from compose and x-bake
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax b438032a60
bake: support compose build cache_to
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax 5ce3909c48
bake: fix compose consistency check
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax 432c2b2650
bake: better handling of compose extension interface
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
Tõnis Tiigi 91fec23f5d
Merge pull request #1153 from crazy-max/update-compose-go
bake: support compose build tags and secrets env
3 years ago
CrazyMax 6cb1b85d7b
bake: support compose build tags
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax e0350f671a
bake: support compose secrets env
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax 1807cfdd26
docs: changes to bake file definition guide
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax 1cb1ee018b
build: enhance warning message when no output specified
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
Justin Chadwell 03885ec9f1 hclparser: strip out blocks for json files
Body.JustAttributes cannot distinguish between blocks and attributes for
JSON files, so the variable block could be included in the list of
attributes returned.

This patch ensures that JSON and HCL files behave the same way by
removing all known block types first, from the provided config schema
and then from a generated definitions schema.

Fixes #1051

Signed-off-by: Justin Chadwell <me@jedevc.com>
3 years ago
Tõnis Tiigi 6a702ebe5b
Merge pull request #1025 from crazy-max/bake-merge-jsons
bake: merge targets and vars from multiple JSON files
3 years ago
CrazyMax c0f8a8314b
bake: support compose build secrets
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax cad7ed68be
bake: merge vars from multiple JSON files
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax c317ca1e95
bake: merge targets from multiple JSON files
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax 243b428a58
compose: add test for port mapping
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax 0b8dde1071
bake: fix skipped group when already visited by another one
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
Sebastiaan van Stijn fc7ba75fd7
Remove uses of deprecated io/ioutil
The package has been deprecated since Go 1.16: https://go.dev/doc/go1.16#ioutil

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
3 years ago
Tonis Tiigi 91e550b715 bake: add path validation for remote bake invocations
This is a stopgap before proper entitlements support
is implemented.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
3 years ago
Tonis Tiigi 280c008f81 bake: make named contexts relative to remote bake input
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
3 years ago
Tonis Tiigi 32f6358d78 bake: add no-cache-filter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
3 years ago
Tonis Tiigi 0fc2b5ca85 bake: add named contexts keys
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
3 years ago
CrazyMax c74b2fe7a4
bake: restrict target name
This fix adds a restriction `[a-zA-Z0-9_-]+`
for target name. This is pretty much the same as the
container name restriction in moby.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax 0044c28b1f
bake: keep target inheritance
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax 5d715ada96
compose: resolve build args from service environment
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax 3400fa5628
compose: test env_file
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax d95ebef55c
bake: fix group resolution
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
Tõnis Tiigi 33c121df01
Merge pull request #881 from crazy-max/fix-bake-print
bake: fix groups print
3 years ago
CrazyMax 4466a24f9e
bake: fix groups print
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax ec9daba87e
bake: ignore NetworkMode field for json and hcl
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
Tõnis Tiigi 7371dda7a2
Merge pull request #863 from zachary-povey/support_host_network_in_compose
Add NetworkMode to bake target
3 years ago
Zachary Povey 7f8dbf890d Remove support for network override in bake
Signed-off-by: Zachary Povey <zachary.povey@autotrader.co.uk>
3 years ago
Zachary Povey 4ee7f70400 Remove NetworkMode support for HCL targets
Signed-off-by: Zachary Povey <zachary.povey@autotrader.co.uk>
3 years ago
Eliott Wiener dcabc22072 bake: build definition file via stdin
closes #833

Accept bake build definition file from stdin with `-f -`.

Signed-off-by: Eliott Wiener <eliottwiener@gmail.com>
3 years ago
Zachary Povey ae53101e89 Add NetworkMode to bake target
Allows specification of network mode in a bake target.

Fixes #848

Signed-off-by: Zachary Povey <zachary.povey@autotrader.co.uk>
3 years ago
CrazyMax 316ca972b6
bake: fix print output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax 82b212bddf
bake: fix protocol detection
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
Tonis Tiigi 7a7b73c043 bake: restore previous override merge behavior
For array fields, overrides are merged together
but override is not merged with the target. If merging
with target is desired we can add support for
overrides with += operator in the future.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
3 years ago
Tonis Tiigi 0e4d7aa7a9 bake: add test for merging overrides
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
3 years ago
Tonis Tiigi 6fc906532b bake: fix using push override with output definition
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
3 years ago
Tonis Tiigi 422ba60b04 use long-running context for client initialization
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
3 years ago
Tõnis Tiigi cb185f095f
Merge pull request #721 from crazy-max/compose-ext
bake: `x-bake` extension field with compose
3 years ago
CrazyMax 89e126fa60
bake: `x-bake` extension field with compose
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax c685e46609
bake: print default group
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
CrazyMax 9aa8f09f14
Set `ConfigFile` to parse compose files with bake
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
3 years ago
Tonis Tiigi 1bb425a882 bake: allow BAKE_CMD_CONTEXT builtin var
Allows accessing the main context for bake command from bake
file that has been imported remotely.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
CrazyMax ba443811e4
Use compose-spec parser
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
4 years ago
Tonis Tiigi 79433cef7a bake: fix target merge between compose and hcl
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tonis Tiigi c5eb8f58b4 bake: new hclparser package
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tõnis Tiigi 03b7128b60
Merge pull request #575 from tonistiigi/user-func-vars
bake: allow user functions in variables and vice-versa
4 years ago
Tonis Tiigi df7a318ec0 bake: allow user functions in variables and vice-versa
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Akihiro Suda 399df854ea
build: split buildflags package
Planned to be imported by nerdctl in future.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
4 years ago
Alex Couture-Beil eec843a325 include default ssh socket when given an ssh-based git url
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
4 years ago
Tonis Tiigi 83868a48b7 temp local copy of userfunc
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tonis Tiigi f036bba48c bake: add test for json vars and attributes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tonis Tiigi 0fe2ce7fac bake: allow attributes in global scope
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tonis Tiigi 33f25acb08 bake: allow variables to reference each other
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tonis Tiigi 0e9066f6ed bake: fix hcl tests layout
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tonis Tiigi 0e9d6460db bake: allow variables across files
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tonis Tiigi dba79ba223 update lint to go1.16/golangci
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tonis Tiigi 7878f0c514 bake: allow variables in user functions
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Felipe Santos f1f464e364 Add setproduct function to the bake HCL
As explained in the following link, it's a very useful function.

https://www.terraform.io/docs/configuration/functions/setproduct.html#finding-combinations-for-for_each
Signed-off-by: Felipe Santos <felipecassiors@gmail.com>
4 years ago
CrazyMax 96e7f3224a
Extend hcl2 support with more functions
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
4 years ago
Tonis Tiigi 40fad4bbb5 progress: make sure all channels have written before returning
Possible write on closed channel on cancellation before.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago
Tonis Tiigi 5bf2ff98c9 bake: support filenames without suffix
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
4 years ago