Skip to content

feat(s3): support SSE-C customer-provided keys - #1017

Open
schaurian wants to merge 11 commits into
cloudnative-pg:mainfrom
schaurian:feat/s3-sse-c
Open

schaurian wants to merge 11 commits into
cloudnative-pg:mainfrom
schaurian:feat/s3-sse-c

Conversation

@schaurian

@schaurian schaurian commented Jul 19, 2026 •

Copy link
Copy Markdown

Support S3 Server-Side Encryption with Customer-provided keys (SSE-C) in the ObjectStore resource. S3-compatible stores such as Hetzner Object Storage and some Ceph setups only offer SSE-C, so the bucket-managed encryption field cannot be used there.

The new sseCustomerKey field of s3Credentials is defined in the barman-cloud library and reaches the plugin through the embedded configuration. It references a secret holding a base64-encoded 256-bit AES key. The library keeps one key file per secret and passes it to every barman-cloud command that accepts it. barman-cloud-check-wal-archive only lists objects and rejects the option, so it does not get the key. The instance Role gets access to the key secret through the library helper SecretNames(), which replaces the plugin's own list of credential secrets.

An ObjectStore that sets sseCustomerKey together with data.encryption or wal.encryption is rejected at admission with a CEL rule, because barman-cloud refuses that combination at run time. The docs get a new section on SSE-C, including the need to keep the key safe and unchanged.

The e2e suite covers a replica cluster whose source and replica object stores use different SSE-C keys, each in its own secret. It checks that every object in each bucket is readable with its own key only.

Closes #646


Merge barman-cloud 0.7.0 before merging this PR.

@schaurian
schaurian marked this pull request as ready for review August 9, 2026 16:14
@schaurian
schaurian requested a review from a team as a code owner August 9, 2026 16:14
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. enhancement New feature or request go Pull requests that update go code labels Aug 9, 2026
@andrein

andrein commented Sep 8, 2026

Copy link
Copy Markdown

Tested at adb7045 with main merged in for Barman 3.20.0, against a Ceph RGW
object store that supports only SSE-C, with CloudNativePG 1.30.0 and
cloudnative-pg/barman-cloud#284 at a300841.

Two changes were required:

  1. sseCustomerKey is not collected by CollectSecretNamesFromCredentials, so
    the instance Role does not cover a key stored in its own Secret, as in the
    aws-sse-c docs example, and credential resolution fails with a forbidden
    error. fix: grant the instance role access to the SSE-C key secret schaurian/plugin-barman-cloud#1 against this branch adds it with a test.
  2. The branch is based on v0.14.0, whose sidecar pins Barman 3.19.1;
    --sse-customer-key exists from 3.20.0, pinned on main since v0.15.0.

With those:

  • key materialised in the sidecar and passed to every barman-cloud command,
    including backup-list and backup-delete
  • WAL archiving continuous; objects report Encryption: SSE-C, 400 without the key
  • base backup completed with Backup.status populated
  • ObjectStore.status.serverRecoveryWindow populated
  • retention applied on the primary
  • recovery into a new Cluster through externalClusters[].plugin reproduced
    the source database

@schaurian

Copy link
Copy Markdown
Author

Thanks for the end-to-end run and the RBAC fix, @andrein — merged schaurian#1 into this branch.

Updated the branch (eed228a):

  • Rebased onto main, so the sidecar carries Barman 3.20.0 and the branch no longer conflicts.
  • go.mod replace now points at the current head of feat(s3): support SSE-C customer-provided keys barman-cloud#284 (14f89b6), which includes your ValidateBackupConfiguration change.
  • CRD, manifest.yaml and API docs regenerated with the repo's dagger tasks; the diff against main is exactly the new field.
  • Docs: the SSE-C section no longer claims the field is independent of encryption — it now states that sseCustomerKey cannot be combined with data.encryption / wal.encryption — and the version note points at Barman ≥ 3.20.0 (sidecar since v0.15.0) instead of the barman issue.
  • Added the new docs words to .wordlist.txt; spellcheck was the other reason the first CI run failed.

Local task ci on this head: commitlint, spellcheck, go test, uncommitted-drift check, docusaurus build and the ephemeral e2e suite (10 passed, 0 failed, 4 skipped — the GCS specs) all pass. golangci-lint reports only gomoddirectives for the temporary replace, which disappears once barman-cloud#284 is released and the replace is dropped.

One observation: the plugin does not call ValidateBackupConfiguration anywhere, so the new barman-cloud check does not currently guard ObjectStore objects — an object store with both sseCustomerKey and data.encryption is still accepted and fails at the first backup. The docs describe that behaviour. If you'd like it rejected at admission, I'm happy to wire the validation into the plugin in a follow-up.

@armru

armru commented Sep 30, 2026

Copy link
Copy Markdown
Member

I pushed a few commits on top:

  • main merged in, so the e2e runs on RustFS;
  • the replace bumped to the new head of feat(s3): support SSE-C customer-provided keys barman-cloud#284;
  • CEL rules on ObjectStore that reject sseCustomerKey together with data.encryption / wal.encryption at admission, with the docs updated;
  • a replica cluster e2e where the source and the replica object stores use different SSE-C keys, each kept in its own secret. It checks that every object in each bucket is readable with its own key only.

That e2e is what found the check-wal-archive problem fixed in cloudnative-pg/barman-cloud#284: --sse-customer-key no longer reaches that command, and the combination with encryption is now rejected at admission instead of failing at run time. The PR description could be updated on both points.

Comment thread manifest.yaml Outdated
Comment thread config/crd/bases/barmancloud.cnpg.io_objectstores.yaml Outdated
schaurian and others added 3 commits October 16, 2026 12:00
Surface the new `sseCustomerKey` field on `s3Credentials` through the
ObjectStore CRD so users can enable Server-Side Encryption with
Customer-provided keys (SSE-C). This is required by S3-compatible
providers that only support SSE-C for encryption at rest, such as
Hetzner Object Storage.

The field flows through the embedded BarmanObjectStoreConfiguration
from the barman-cloud library, so this change is limited to bumping the
dependency, regenerating the CRD and the consolidated manifest, and
documenting usage in the object stores guide.

Depends on cloudnative-pg/barman-cloud#284 (temporarily pinned via a
replace directive until that change is released).

Closes cloudnative-pg#646

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Florian Schauer <florian@schauer.to>
The sidecar reads credential secrets through a Role whose resource
names are collected from the credential references. Without the new
reference in that list, a key kept in its own secret, as in the docs
example, is forbidden and every barman-cloud command fails.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Andrei Nistor <andrei@nistor.tech>
Point the temporary barman-cloud replace at the head of
cloudnative-pg/barman-cloud#284 that keeps each SSE-C key in its own
file. A replica cluster archives WAL to its own object store while
restoring it from the source one in the same sidecar, and with a single
shared file each operation could run with the key of the other object
store.

Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
@mnencia mnencia changed the title feat: support S3 SSE-C (customer-provided encryption keys) feat(s3): support SSE-C customer-provided keys Oct 16, 2026
@mnencia
mnencia force-pushed the feat/s3-sse-c branch 2 times, most recently from 09e0320 to 7d4f06a Compare October 16, 2026 10:26
armru and others added 5 commits October 16, 2026 14:21
The barman-cloud CLI refuses --sse-customer-key together with
--encryption, so an ObjectStore setting both sseCustomerKey and data or
wal encryption was accepted and then failed at its first backup or WAL
archive. Reject it when it is created or updated instead.

Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
Run the replica cluster scenario with the source and the replica object
stores each encrypted with its own SSE-C key kept in its own secret. The
designated primary archives WAL to one object store while restoring it
from the other in the same sidecar, and the switchover exercises both
directions.

At the end, check that every object in each bucket is readable with its
own key only, and that an ObjectStore combining SSE-C with bucket-managed
encryption is rejected.

Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
Point the temporary barman-cloud replace at the head of
cloudnative-pg/barman-cloud#284 that stops passing --sse-customer-key to
barman-cloud-check-wal-archive. That command rejects the option, and it
runs before the first WAL file is archived, so an object store with an
SSE-C key could never start archiving. The SSE-C e2e test caught it.

Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
Point the temporary replace at the current head of barman-cloud#284,
which lists the secrets referenced by the credentials. Use it instead of
the local list, so a new credential field cannot be left out of the
instance role. Keep a role test for the SSE-C key secret.

Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
Pick up the new description of sseCustomerKey, and the fields that main
already carries in the barman-cloud library.

Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
mnencia added a commit to cloudnative-pg/barman-cloud that referenced this pull request Oct 16, 2026
Add an optional `sseCustomerKey` field to `S3Credentials`. It references
a secret with a base64-encoded 256-bit AES key. S3-compatible stores
such as Hetzner Object Storage and some Ceph setups only offer SSE-C, so
the existing bucket-managed `encryption` field cannot be used there.

The key is written to `/controller/.sse-customer-keys/<secret>/<key>`
before the auth method is handled, so it works with every auth method,
including `inheritFromIAMRole`. The path depends on the secret and key
because one process can serve several object stores at once, for example
a replica cluster archiving to its own store while restoring from the
source one. A shared file would let a command pick up the wrong key. Key
files are not removed when the field or the secret goes away. Changing
the key makes existing objects unreadable, so a new key needs a new
`destinationPath` or `serverName`.

The key is passed as `--sse-customer-key file://<path>` to every
barman-cloud command except `barman-cloud-check-wal-archive`, which only
lists objects and rejects the option. `ValidateBackupConfiguration`
rejects `sseCustomerKey` together with `data.encryption` or
`wal.encryption`, which barman-cloud refuses at run time.

The credentials types now list the secrets they reference through
`SecretNames()`, with a test that fails when a new secret field is not
listed. Consumers can use it to grant access to those secrets instead of
keeping their own list.

Requires Barman 3.20.0 or later. The plugin side is
cloudnative-pg/plugin-barman-cloud#1017.

Refs cloudnative-pg/plugin-barman-cloud#646

Assisted-by: Claude
Signed-off-by: Florian Schauer <florian@schauer.to>
Signed-off-by: Andrei Nistor <andrei@nistor.tech>
Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
Co-authored-by: Andrei Nistor <andrei@nistor.tech>
Co-authored-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
Co-authored-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
Drop the temporary replace. The library merged cloudnative-pg#284, and the release
that carries it is not out yet.

Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
@mnencia
mnencia force-pushed the feat/s3-sse-c branch 2 times, most recently from 5d91b7c to 7db3aa8 Compare October 16, 2026 13:11
Replace the em-dashes, split the long closing paragraph, and say that
a new key needs a new destinationPath or serverName.

Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
Call SecretNames on the configuration, as the embedded BarmanCredentials
field is promoted.

Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request go Pull requests that update go code size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature request] Support for S3 SSE-C - Server-Side Encryption with Customer-provided keys

5 participants