feat(sudoers): let supabase-admin-agent invoke pgbackrest reconcile as root

[hunleyd]

Summary

The salt subcommand's systemd service runs as the unprivileged supabase-admin-agent OS user, which can't write to /etc/pgbackrest/conf.d (owned pgbackrest:postgres 02770). This PR adds a NOPASSWD sudoers entry scoped to exactly pgbackrest reconcile, and changes /opt/supabase-admin-agent ownership to root:root so that sudo grant can't be used to elevate a binary the unprivileged agent user could otherwise replace first.

(bot-generated information collapsed below)

---

Warning

Dependencies

1. Requires supabase/supabase-admin-agent#112 merged (adds the pgbackrest reconcile command this sudoers entry grants access to)
2. Requires INDATA-1151 resolved via supabase/platform#37380 (the platform repo's provisioning scripts currently revert the root-ownership change on every instance boot, so this must land before the sudoers rule is safe on real instances)
3. Requires INDATA-1171 resolved via supabase/platform#37381 (removes drift between the platform repo's own sudoers copies and the AMI-shipped one; per #37381 itself, this closes the boot-to-first-converge window but is not what makes reconcile's sudo check succeed in steady state. See #5 below.)
4. Requires INDATA-1312 resolved (not yet started). supabase/salt's own init.sls reasserts supabase-admin-agent:supabase-admin-agent ownership on every converge (~10 min cadence, fleet-wide) and grants the binary cap_fowner/cap_chown capabilities that bypass ownership restrictions entirely regardless of chown. Merging Create LICENSE #1-Automate our build with Github Actions #3 above establishes root ownership only in the window between an instance's own boot and its first Salt converge; this dependency is what makes it durable. Also flagged directly on supabase/salt#826, an in-review PR that currently reinforces agent ownership rather than fixing it.
5. Requires INDATA-1186 resolved via supabase/salt#757 (in progress, unmerged). reconcile only ever runs immediately after a Salt state apply in the same process, and Salt's own file.managed state for /etc/sudoers.d/supabase-admin-agent (replace: True, sourced from a copy tracked in the salt repo, separate from both Automate our build with Github Actions #3's platform copies and this PR's AMI copy) always rewrites the file on that same converge before reconcile ever checks sudo. This is the dependency that actually determines whether reconcile's sudo check succeeds in production; Create LICENSE #1-Support for more images #4 above are necessary but not sufficient without it.

---

Details

• Adds one NOPASSWD sudoers entry so the salt cycle can elevate just pgbackrest reconcile via sudo, mirroring the existing salt-call entry in the same file. No broader group membership needed.
• Changes /opt/supabase-admin-agent (directory, unpacked binary) from owner supabase-admin-agent to owner root:root, so the sudoers grant above cannot be used to elevate a binary the unprivileged agent user could replace first. Pins explicit mode: "0755" on both so this doesn't depend on umask or the release tarball's stored permission bits. (The symlink itself carries no separate owner/group. state: link with the file module's default follow: yes would only re-apply those to the link target, already covered by the unpack task above; the symlink still gets mode: "0755" explicitly.)

Why this is safe

• Scoped exactly to one command. The sudoers rule matches the full literal path /opt/supabase-admin-agent/supabase-admin-agent pgbackrest reconcile. It can't be used to run anything else as root. Confirmed against the actual call site in supabase-admin-agent PR Generate optimized config for PG #112: reconcileBin is a hardcoded constant equal to this exact path, with no --config flag and no os.Executable() resolution that could drift from it.
• Root ownership blocks binary tampering, with known gaps. The ownership change above is what stops the agent user from swapping the binary it's allowed to sudo-run as root. However, this AMI-time ownership is currently undone at instance boot: the platform repo's provisioning scripts (init-scripts/project/00-init.sh and init-scripts/suspend-and-wake/00-init.sh) recursively chown the whole directory back to the unprivileged agent user after every install (see INDATA-1151 in Dependencies above). Even once that's fixed, supabase/salt reasserts agent ownership on its own converge cadence and separately grants the binary cap_fowner/cap_chown capabilities that bypass ownership restrictions entirely; see INDATA-1312 in Dependencies above. The root-ownership property this PR relies on for safety is not durable in production until INDATA-1312 also lands.
  • That same platform provisioning flow is also what writes /opt/supabase-admin-agent/config.yaml (root templates it, then chown supabase-admin-agent:supabase-admin-agent + chmod 0640, per init-scripts/{project,suspend-and-wake}/00-init.sh), and a periodic salt state (salt/state/formula/supabase-admin-agent/init.sls) re-templates it at 0644 running as the agent user. reconcile reads this file (--config, default /opt/supabase-admin-agent/config.yaml) and derives EffectiveAdminAPIUser() from it, so even with the binary root-owned, root-run reconcile still trusts a config the agent user can write. INDATA-1151's fix needs to explicitly decide config.yaml's final ownership (root-own it too, vs. carve it out as agent-writable for the salt refresh path); it isn't automatically solved by scoping down the recursive chown on the binary/dir alone.
  • audit-specs/baselines/prod-deployed/files-opt-deployed.yml and baseline.yml still record /opt/supabase-admin-agent/README.md and the binary as owner 993/group 987 (the agent uid/gid). Because the platform boot scripts above chown -R real instances back to that uid/gid on every project creation and wake, those baselines stay accurate on live instances even after this PR merges. They only need updating once INDATA-1151 actually removes that revert. Left untouched here; update them as part of INDATA-1151 instead.
• A second, separate revert path, not covered by INDATA-1151. The platform repo also ships its own copy of this sudoers file at init-scripts/suspend-and-wake/supabase-admin-agent.sudoers.conf, containing only the pre-existing salt-call/gpg entries. setup_supabase_admin_agent() in init-scripts/suspend-and-wake/00-init.sh unconditionally reinstalls that copy to /etc/sudoers.d/supabase-admin-agent on every project wake, silently stripping the pgbackrest reconcile line this PR adds. The agent only logs the resulting sudo denial at Warn, so this fails silently. See INDATA-1171 in Dependencies above.
• Same pattern already in production. supabase-admin-agent already elevates salt-call and gpg via this exact mechanism (pkg/salt/exec.go); this isn't a new privilege model, just one more entry.
• Syntax validated. visudo -c parses the file clean.

Alternatives considered

• Group membership instead of sudo. /etc/pgbackrest/conf.d is pgbackrest:postgres 02770. The writable group is postgres, not a pgbackrest-specific group. setup-postgres.yml runs initdb with --allow-group-access, so the postgres group has read access to the entire live PGDATA (every table file, every WAL segment) on every instance. Adding supabase-admin-agent to that group to reach conf.d would grant a much larger blast radius than the single-argv-path sudo grant this PR adds. Rejected in favor of keeping the narrowly-scoped sudo rule.

---

Testing

• visudo -c -f ansible/files/supabase_admin_agent_config/supabase-admin-agent.sudoers.conf: parsed OK
• visudo -c -f ansible/files/adminapi.sudoers.conf: parsed OK
• Manual: once all three dependencies land, confirm the salt cycle's sudo-invoked pgbackrest reconcile succeeds on a test instance without a password prompt, and that /opt/supabase-admin-agent stays root-owned after a fresh project creation and a pause/wake cycle

Smoke Test Results

PASS: test-postgres-pitr.sh (engine 17)

Command: ./scripts/test-postgres-pitr.sh 'https://github.com/supabase/postgres/pull/2291' --engine 17

The script itself reported a failure at the final health-check step because the invocation was initially missing an unrelated CLI credential (SUPADEV_SYSTEM_API_KEY). Backup creation and PITR restore both succeeded before that point. Re-running just the health check (supadev project-health <restored-project>) once the credential was set confirmed all components ACTIVE_HEALTHY (Storage, Realtime, Postgres (Admin), Postgres (User), Supavisor (TXN), PgBouncer, and others). Net result: PITR backup/restore works correctly with this PR's ansible ownership change.

[INFO] Filtered to single engine: 17
============================================================
============================================================
[INFO] Testing PostgresEngines: 17
[INFO] Using region: us-east-1
[INFO] Project prefix: supadev-pitr-1784744543
[INFO] GitHub PR URL: https://github.com/supabase/postgres/pull/2291
============================================================
============================================================
[INFO] Source project name: supadev-pitr-1784744543-pg17
[INFO] Restored project name: supadev-pitr-1784744543-restored-pg17
[INFO] Region: us-east-1
[INFO] Start time: 2026-07-22 14:22:23
[INFO] Using version from PR: https://github.com/supabase/postgres/pull/2291
[INFO] Resolved version: 17.6.1.151
[STEP] Checking organization tier...
[INFO] Organization is already on tier_pro
[STEP] Creating project supadev-pitr-1784744543-pg17 with PITR enabled...
[SUCCESS] Project supadev-pitr-1784744543-pg17 created with PITR enabled
[INFO] Time to ACTIVE_HEALTHY: 373 seconds
[STEP] Waiting for physical backup data (30s sleep, then polling)...
[INFO] Polling attempt 1/20...
[SUCCESS] Physical backup data found
[STEP] Extracting earliest PITR backup timestamp...
[INFO] Earliest PITR backup: 2026-07-22 14:28:42
[STEP] Restoring PITR backup to new project supadev-pitr-1784744543-restored-pg17...
[SUCCESS] PITR backup restored to supadev-pitr-1784744543-restored-pg17
[INFO] Time to restore: 343 seconds
[STEP] Checking health of restored project...
[ERROR] Restored project health check failed (missing SUPADEV_SYSTEM_API_KEY in this invocation — see note above)

--- follow-up: supadev project-health supadev-pitr-1784744543-restored-pg17 (credential fixed) ---
  Storage:            healthy, ACTIVE_HEALTHY
  Realtime:           healthy, ACTIVE_HEALTHY
  Postgres (Admin):   healthy, ACTIVE_HEALTHY
  Postgres (User):    healthy, ACTIVE_HEALTHY
  Supavisor (TXN):    healthy, ACTIVE_HEALTHY
  PgBouncer:          healthy, ACTIVE_HEALTHY

SKIPPED: test-postgres-pause-restore.sh (engine 17)

Command: ./scripts/test-postgres-pause-restore.sh 'https://github.com/supabase/postgres/pull/2291' --engine 17

Blocked by a pre-existing local dev-stack defect ("IPv6 address is not defined for this project", 500 from mgmt-api) unrelated to this PR's diff (ansible sudoers/ownership only). OrbStack's ipv6 config was already correct, so the fix requires an engine restart that drops every container on the host. Skipped rather than disrupt other work; PITR is the test most directly relevant to this diff's pgbackrest/archiving change and passed cleanly.

[INFO] Filtered to single engine: 17
============================================================
============================================================
[INFO] Testing PostgresEngines: 17
[INFO] Using region: us-east-1
[INFO] Project prefix: supadev-pause-1784745764
[INFO] GitHub PR URL: https://github.com/supabase/postgres/pull/2291
============================================================
============================================================
[STEP] Resolving version from PR for engine 17...
[SUCCESS] Resolved version for 17: 17.6.1.151
[STEP] Inserting version 17.6.1.151 as ga...
[SUCCESS] Version 17.6.1.151 inserted as ga
============================================================
============================================================
[INFO] Project name: supadev-pause-1784745764-pg17
[INFO] Region: us-east-1
[STEP] Checking organization tier (pause/restore requires Free tier)...
[INFO] Already on free tier, no changes needed
[STEP] Creating project with PostgresEngine 17...
[ERROR] Failed to create project supadev-pause-1784745764-pg17 with PostgresEngine 17
  (root cause: "IPv6 address is not defined for this project" — local dev stack defect, not this PR)

---

Misc

Related issues: INDATA-996 (this PR is the sudoers/ownership plumbing pgbackrest reconcile needs, not the tier-gating logic itself)

Also carries an unrelated, stacked fix for INDATA-1153: grants adminapi sudo for a new pgbackrest ensure-spool-dir subcommand (ansible/files/adminapi.sudoers.conf), the companion commit for which lives in supabase-admin-agent#120 (which this branch's own dependency, INDATA-996, is stacked under). enable, setup-replica, and stanza-upgrade all need this grant to backfill /data/pgbackrest_spool on instances whose data volume predates postgres#2312's AMI directory-creation task. Riding this PR purely because #120 depends on this branch; unrelated to the pgbackrest reconcile sudoers work this PR is otherwise about.

[hunleyd]

:robot_face: Minor comment-wording issue found while reviewing the platform-repo sudoers sync (supabase/platform#37381), which copies this PR's sudoers lines byte-for-byte.

The added comment's contrast clause is imprecise:

# pgbackrest reconcile writes /etc/pgbackrest/conf.d (pgbackrest:postgres 02770) and the agent is in neither group, so this subcommand needs root; the salt subcommand above stays unprivileged

"the salt subcommand above stays unprivileged" reads as if it's describing the entry directly above it (NOPASSWD: /usr/bin/salt-call) — but that entry is a root grant (salt-call runs as root). What actually stays unprivileged is the agent's own salt subcommand process that invokes salt-call via that grant, not the grant itself. It's also a positional reference ("above"), which drifts if the file's line order ever changes.

Since supabase/platform#37381 copies this comment verbatim to keep the two copies byte-identical, a wording fix here would need to land in both places together. Suggested rewording: "the agent's own salt subcommand keeps running as the unprivileged supabase-admin-agent user." Not blocking — just flagging so it doesn't propagate further as more copies of this file get created (three platform-repo copies already exist).

[hunleyd]

@claude review