Stop uploading raw calibration logs as public Actions artifacts

[MaxGhenis]

Security fix, held for Max (cos d1596). By default this would land with the d833 uk-data batch. The decision asks whether it should land on its own, before that batch.

Why

Two raw calibration logs, constituency_calibration_log.csv and la_calibration_log.csv, are uploaded as Actions artifacts by two kinds of build. One is every pull request build that gets as far as the upload steps. The other is every main build the push workflow runs: it triggers on pushes that change pyproject.toml, and its gate can skip the build. They hold every target's weighted estimate and value at every logged epoch, nationally and for each constituency and local authority, with no contributor count, dominance check or suppression.

Sol's review of #599 showed the risk on synthetic inputs. An income category supported by one household publishes a weighted income and a weighted count, and dividing them gave back that household's income exactly. A large weighted count does not protect it. That was a demonstration on synthetic data; no disclosure has been observed in a real log.

This repository is public, so its Actions artifacts can be downloaded by anyone signed in to GitHub. As of 15 October 2026 there are 448 live artifacts of this kind:

Artifact	Live	Created	Expire by
constituency_calibration_log.csv	223	19 Jul – 15 Oct 2026	13 Jan 2027
la_calibration_log.csv	223	19 Jul – 15 Oct 2026	13 Jan 2027
base_constituency_calibration_log.csv (from #599's early runs)	2	15 Oct 2026	13 Jan 2027

Of the first two kinds, 18 each come from pushes to main. These counts come from gh api repos/PolicyEngine/policyengine-uk-data/actions/artifacts?name=<name> on 15 October; the independent review could not repeat that query from its sandbox.

Nothing in this repository downloads these artifacts or reads them later, and no documentation tells people to fetch them. git grep calibration_log finds the code that writes the logs, the upload steps and, after this PR, the guard. Consumers outside the repository can't be ruled out.

What changes

• pull_request.yaml and push.yaml no longer upload the two logs. The builds still write them, and they stay on the runner.

• Both data jobs set save-cache: false on setup-uv. They still restore uv's package cache, but never save one. setup-uv's cache is on by default on hosted runners, and a later step can move its location into the workspace through UV_CACHE_DIR.

• tests/test_workflow_artifacts.py is a regression check on the routes out of jobs that can reach the data. A job counts as one when it, or its workflow, does any of these:

  • mentions secrets in any form or case other than secrets.GITHUB_TOKEN: secrets.NAME, SECRETS.NAME, secrets['NAME'] or toJSON(secrets). Expressions are case-insensitive, and each is presumed a data credential, whatever it's called;
  • passes secrets on;
  • can request an OIDC token (id-token: write, or permissions: write-all);
  • signs in to Google Cloud;
  • downloads artifacts.

  In such a job:

  • every action must be reviewed for the inputs it's given. For example, setup-uv can't be pointed at the workspace, and it must not save its cache;
  • every artifact upload must name literal files reviewed for that job. Directories, globs, exclusions and expressions fail, and every line of a multi-line path is checked;
  • each reviewed file needs its producing command in an earlier run step, not on a comment line;
  • every run line that matches a known publishing command must be, in full, a command reviewed for that job, with backslash continuations joined. Known publishing commands are any gh command, git push, any curl or wget, aws s3, gsutil, gcloud, Hugging Face uploads and twine. Every curl and wget counts, not only their upload options, because each review round found another spelling that sends a file (-sT, -H @file, --url-query name@file, --variable name@file with --expand-url). No uk-data workflow runs either, so a new use is reviewed like any other command. The reviewed ones are push.yaml's make upload (the private Hugging Face and GCS datasets) and the release tag;
  • the versioning job's commit is reviewed with its exact inputs. Because it adds the whole workspace, that job may run only its three reviewed lines: the towncrier install, the version bump and the changelog build. It also may not set a step shell, env or working-directory, or a job or workflow env, defaults or container, any of which could run something around those lines. A bare make would build the data, so command names aren't enough;
  • a reusable workflow fails.

  Jobs that can't reach the data (docs, changelog) may publish.

  What it doesn't establish, as the docstring says: it works from secret, action and command names, and it does not inspect:

  • scripts, composite actions, reusable workflows, containers or services;
  • shell expansion;
  • step summaries or job outputs.

  A producer pattern shows text in an earlier step. It doesn't show that the step ran, that the path is a regular file, or that the contents are disclosure-controlled. Changes to these routes need human review.

  Mutation tests add each of Sol's bypasses to a clean synthetic data job, including:

  • workspace cache settings on setup-uv;
  • make upload; curl …, and a commented reviewed command;
  • a workflow-level credential;
  • an inherited secret and an arbitrarily named one;
  • a producer only in a comment, in a disabled step, or after the upload.

  As a positive control, the check fails at origin/main on the two raw-log uploads, and on setup-uv saving its cache in both data jobs.

The artifacts that already exist are not touched. Deleting them is a separate, irreversible action, and it is part of the same decision.

Relation to #599

#599 (the W3 fit comparison) also removes the pull request uploads. It replaces them with a disclosure-controlled fit summary: within-10% booleans, plus an error only for national targets supported by at least 10 households. Whichever lands second needs a conflict resolution in pull_request.yaml. If this lands first, #599, #600 and #602 will fail this guard until they add their reviewed entries:

• each summary with its producer: fit_summary.json, base_fit_summary.json, the distribution reports and the aggregate results;
• the comparison job's comment posting (gh api).

That is intended: each new route out of a data job gets a reviewed line. This PR exists so the leak can stop before the d833 batch.

Tests

pytest policyengine_uk_data/tests/test_workflow_artifacts.py: 93 passed here. Against origin/main's workflows, the two real-workflow checks fail, as expected.

axiom: n/a: CI artifact handling; no policy change

🤖 Generated with Claude Code