add rego integration to source policies

[tonistiigi]

This is an early version of rego format support for defining source policies. Not ready for merge but suitable for early testing and feedback.

Currently many untested parts and unimplemented fields/builtins. I also discovered a bug with git/http metadata resolve in BuildKit that needs to be fixed for support of some fields.

For input schema see policy/types.go atm until we expose it better.

For app.Dockerfile, matching app.Dockerfile.rego is loaded. Only local files supported atm. This would be extended with manual control via flags and helper commands for testing.

Set export BUILDX_POLICY_DEBUG=1 and use plain progress mode to see the internal input data and policy decisions. This is a temporary debug until there is better progressbar integration.

Some examples in https://gist.github.com/tonistiigi/a8a1fdf39796ba484a31af18afb04bfc https://github.com/tonistiigi/buildx-rego-examples . BuildKit v0.26 is needed.

@crazy-max @dvdksn @cpuguy83 @colinhemmings

[dvdksn]

Encountered this:

FROM alpine

package docker

default allow := false

allow if input.local

allow if {
  input.image.hasProvenance
}

decision := {
    "allow": allow 
}

ERROR: failed to build: failed to solve: alpine: failed to resolve source metadata for docker.io/library/alpine:latest: could not resolve image due to policy: too many policy requests

[tonistiigi]

@dvdksn should be fixed as part of moby/buildkit#6383 https://gist.github.com/tonistiigi/a8a1fdf39796ba484a31af18afb04bfc#file-9-provenance-dockerfile . Although it looks like there is another issue where if the image is not an OCI index, then you get a different error when parsing instead of a policy deny error. Signature keys have not been implemented yet in this PR.

[dvdksn]

Perm would always be 600 I guess - and not sure what UID/GID would show?

[tonistiigi]

Maybe we should remove them. These are options you can set in LLB that set the values for the downloaded file. But these are not exposed in Dockerfile, and if you use smth like COPY --chmod then this is actually different mode that is applied during the copy.

[AkihiroSuda]

nit: err should be returned if err != nil && !errors.Is(err, os.ErrNotExist)
e.g., EACCES

[AkihiroSuda]

Any reason to reject symlink?

[tonistiigi]

Just wanted to make sure there is no breakout here. Normally this shouldn't matter as Dockerfile path symlinks are already resolved and default policy is relative to that resolved path. But I'm ok with symlinks only for policy as well, as long as breakout cases are protected.

[tonistiigi]

Added new features:

• print() for debugging
• import support for loading shared policies (only local context atm)
• load_json() for importing JSON files (only local context atm)
• git signature fields and verification with local key

New examples in https://github.com/tonistiigi/buildx-rego-examples . I needed to move away from gist as it doesn't allow pushing directories.

[tonistiigi]

Added image signature verification example.

[tianon]

For app.Dockerfile, matching app.Dockerfile.rego is loaded.

As a Dockerfile author / content creator, I'm honestly struggling to understand the purpose of this -- why would I, as the content author, want to specify policies for the build of my Dockerfile? As-is, this feels like a fully out-of-band way to add extra constraints to the Dockerfile itself, and I'm not really understanding what the goals are (or why they're all so fully out-of-band).

Is there a design document somewhere that I could read that explains in more detail what problems are being solved?

From the DOI perspective, some kind of policy engine is interesting, since I'd like to apply polices to the Dockerfiles written by third parties / control the "remote content fetching" behavior of BuildKit more explicitly, but not if it's picking up policies written by the content authors. 🤔

[tonistiigi]

As a Dockerfile author / content creator, I'm honestly struggling to understand the purpose of this -- why would I, as the content author, want to specify policies for the build of my Dockerfile?

To protect yourself from your project becoming a target when one of the upstreams gets attacked or becomes malicious. Same reason why we add digests to critical dependencies in Dockerfile, or why Akihiro wants to improve the security of Moby in PRs like https://github.com/moby/moby/pull/51626/files . Except, 1) this is much more powerful than digest check in the image/git ref 2) you don't need to update to a new digest every time there is an update in upstream or you want to build a different configuration with build-arg 3) digests are not human-readable, making them poor to verify in PR review.

feels like a fully out-of-band
since I'd like to apply polices to the Dockerfiles written by third parties

It's not out-of-band or restricted to upstream only. It is part of the project and loaded automatically. Given the amount of functionality in here, it is impossible to inline something like this into Dockerfile itself, and it would make for very poorly readable Dockerfiles. But they are tightly integrated and a policy is written for a specific (set of) Dockerfiles. As they serve different purposes I think it is very useful that they are not mixed together so you can concentrate on a specific aspect while writing each file.

Additionally you can add your own policies when invoking cli if you want specific additional constraints. This would need a new flag that is not implemented yet, but mentioned in the description.

If you are the author of DOI Dockerfile you can write a policy that protects you, your CI and release bot, and all the other 3rd parties either working with your codebase or building their own versions of the artifacts using your Dockerfile. They don't need to use your policy and can opt-out or set their own more restrictive policy, but you as a repo/Dockerfile author can define their secure-by-default experience.