Skip to content

Preserve public course signup totals without registration PII #133

Description

@alexeygrigorev

Product outcome

Restore the copied CMP registration page’s public signup-total row without importing, fabricating, or exposing a registration record.

For one exact registration campaign and its recorded cohort, the public page may render only the existing CMP sentence:

N already registered for 2026 cohort

N is the complete, unrounded non-negative total from one provenance-backed historical baseline plus registrations created by the new website after an explicit non-overlapping cutover boundary. The aggregate never creates a CourseRegistration, learner, account, consent, email, or delivery side effect.

Product and architecture authority

Confirmed behavior and regression cause

At the pinned CMP commit:

  • courses/views/registration.py, SHA-256 b6c4697e7e52adaa04a8087caf7acf72557b10b0a743874332267bdf267e25ad, supplies campaign.registrations.count() as signup_count;
  • courses/templates/courses/register.html, source SHA-256 a70ef892a1f7ded5525b2072bc5538e5503148df56be6664dd1e44b37615a692, renders the row only when signup_count is non-zero;
  • the website view remains source-identical, while the readable accessibility overlay of the copied template is recorded at SHA-256 d5af022a7f451c41a07b7aa643f2ff22ddb30724894df5cea75c54a09b4cdd7c.

Public source observations are time-sensitive: the owner first reported 472, independent #128 QA observed 475, and a read-only check on 2026-08-11 observed 478 for ml-zoomcamp and 1,915 for ai-dev-tools. Those values prove the behavior but are not import authority and must not be hard-coded, scraped from HTML, or copied into fixtures.

The accepted #128 development artifact is deliberately different: exact source SHA-256 ac55cb0cb10cc0924dd8c9a9e63fe9b09ae809cac8aac14d6da2ce46c3586d04 contains two courses_registrationcampaign rows but requires courses_courseregistration to be empty. review_import.manifest.SENSITIVE_TABLES and the development importer’s protected-target checks enforce that boundary. The repository-local CMP databases and sanitized review copies also contain no authoritative registration population. Therefore only an authorized protected CMP database snapshot can supply the real baseline.

Product decision: course-owned aggregate overlay

Do not put a mutable integer on RegistrationCampaign, reuse the event-provider aggregate models, weaken the #128 sanitizer, or create placeholder registrations. The courses app owns a narrower three-part overlay:

  1. CourseRegistrationCountSourceRun
    • UUID; adapter, schema, and count-policy versions;
    • whole protected-snapshot SHA-256 and byte size;
    • schema-contract and canonical aggregate-manifest checksums;
    • a digest of the registered source reference, never its path/name;
    • capture time, operator-attested source freeze/cutoff time, safe campaign/row totals;
    • exact state, revision, actor reference, bounded reason code, and timestamps.
  2. CourseRegistrationCountRevision
    • immutable source-run reference;
    • exact target RegistrationCampaign and current legacy Course (the cohort record until the expand-and-contract rename), plus slug snapshots;
    • baseline count, coverage/cutover boundary, aggregate checksum, state, revision, and timestamps.
  3. CourseRegistrationCountSlot
    • exactly one revision-guarded pointer per (campaign, cohort);
    • active mode baseline_plus_native or rows_only;
    • active baseline revision, native-start instant, public-total revision, and prior pointer evidence needed for rollback.

Source-run state is exactly staged, validated, active, cancelled, rolled_back, or quarantined. Count-revision state is exactly staged, validated, active, superseded, rolled_back, or quarantined. Counts and provenance on a persisted source run/revision are immutable. Uniqueness and database constraints prevent two active contributions for one campaign/cohort slot.

These are target-owned course models/services. Keep them outside the copied courses/models/course.py where practical. A necessary one-line integration in the copied registration view is an explicit recorded overlay; the copied registration template and its visual/accessibility composition remain unchanged.

Protected source and aggregate derivation

Version 1 accepts only an opaque key from a code/configuration-owned COURSE_REGISTRATION_COUNT_SOURCES registry. Studio/API never accept or return an upload, arbitrary server path, database URL, credential, or raw source locator.

The registered source is one immutable, private CMP database snapshot with exact SHA-256, byte size, schema version/checksum, capture time, and freeze/cutoff evidence. The adapter opens it read-only/query-only with bounded resource limits. It validates the exact campaign/course/registration tables, relations, required columns, uniqueness contract, and timestamp awareness, then derives only:

  • COUNT(*) grouped by exact campaign_id and the registration’s recorded course_id;
  • safe campaign/course IDs and public slugs needed for exact target mapping;
  • aggregate minimum/maximum registration timestamps and safe row totals needed to prove the cutover boundary.

The adapter never selects, materializes, hashes, returns, persists, logs, or audits email, normalized email, name, company, country, region, role, comment, account ID, consent/newsletter value, IP, user agent, or any other attendee/member field. It performs no source write, network request, provider action, account creation, registration, enrollment, email, newsletter, or Datamailer/Relay work.

Mapping is exact, not editable or fuzzy: source campaign slug and recorded course/cohort slug must resolve to exactly the target campaign and its current course imported by #128. Null course rows, a campaign spanning more than one cohort, missing/duplicate targets, a changed current-course pointer, rows beyond the declared source cutoff, or any identity/checksum mismatch quarantine the candidate. Numeric primary-key equality alone is not mapping authority.

Import, activation, and replay contract

The shared application service implements the following exact behavior:

  • Empty target: stage one source run and its complete campaign/cohort revisions atomically. Validation must cover the complete manifest; activation publishes all validated slots atomically or none.
  • Exact replay: the same source-reference identity, whole-source/schema/policy/manifest checksums, cutoff, and derived rows returns the original safe result with replayed=true; it creates no row, pointer move, audit duplicate, sequence change, registration, or side effect.
  • Different/partial target: the same immutable source identity with different bytes, schema, count, cutoff, or manifest; a partial prior insert; or a conflicting campaign/cohort identity fails closed and changes nothing. A genuinely newer whole-source checksum creates a new immutable staged run and never silently changes the active total.
  • Activation: confirmed and reasoned, idempotent, revision guarded, all-or-nothing, and retains the prior active revision for rollback. It requires an exact source freeze/cutoff and a strictly non-overlapping native-start boundary.
  • Rollback: confirmed, reasoned, idempotent, revision guarded, and restores the previous complete pointer set without deleting history or registrations.

Dry-run and validation return only safe checksums, counts, public campaign/cohort slugs, states, revisions, and bounded reason codes. A failed new run preserves the last complete active total unless the campaign/cohort identity or coverage boundary itself has become stale.

Public total and no-double-count behavior

A single course application query replaces direct campaign.registrations.count() in the copied view. It returns an integer only when coverage is complete for the campaign and its current cohort.

In baseline_plus_native mode:

public total = immutable baseline
             + target CourseRegistration rows for the exact campaign/cohort
               whose created_at is at or after native_start_at

Activation verifies every baseline source row is strictly before native_start_at, no target row for the slot predates that boundary, and the source was frozen at or before it. Registrations created normally by the website after activation therefore increase the visible total immediately and exactly once. Rows for another campaign/cohort, null/ambiguous course rows, and pre-boundary rows never leak into the number.

The existing #128 development-content import continues to require courses_courseregistration to be empty. No other importer may insert legacy course-registration rows while a slot is in baseline_plus_native mode.

When the complete registration database is later migrated, its owning migration must preserve recorded campaign, course/cohort, and created_at; insert legacy rows and switch the same slot to rows_only in one transaction; prove the pre-boundary imported row count/checksum equals the active baseline; and prove the public total is identical immediately before and after the pointer switch. The aggregate is marked superseded, not deleted. A mismatch, duplicate/overlap, unresolved target, or partial import blocks the switch. Rollback may restore baseline_plus_native, where the timestamp boundary excludes retained imported historical rows and still counts native rows once.

This issue implements and tests the aggregate-to-row replacement transition with synthetic rows, but it does not perform the full PII migration or decide its field/consent/account reconciliation.

Public rendering, missing, stale, and error states

  • Complete total 0: pass integer zero to the existing context; the copied {% if signup_count %} behavior omits the row. Do not display an inferred “0 already registered”.
  • Complete total 1: render exactly 1 already registered plus the existing edition suffix when present.
  • Complete total greater than one: render the exact decimal integer with the same invariant word registered; do not add a new noun, rounding, abbreviation, or marketing copy. The existing source sentence needs no singular/plural noun branch.
  • No slot, no active validated revision, incomplete coverage, quarantined source, stale campaign/cohort pointer, changed registered-source checksum, failed replacement, or aggregate-specific integrity error: return incomplete/no public count and omit the row, never convert the condition to zero.
  • A failed staged refresh leaves a still-valid prior active revision readable. A changed target identity or invalidated coverage makes the old revision stale and omits it until reconciliation.
  • Infrastructure/database failure follows the normal safe site error policy; it is never masked as a trustworthy zero.

The total appears only on the copied registration campaign page. No course catalogue/detail/dashboard, public/compatibility API, SEO metadata, structured data, sitemap, search document, analytics payload, or cache key gains the count or provenance. Registration pages remain private/no-store/noindex/zero-TTL under the existing route policy, so this issue adds no public-cache invalidation machinery.

Studio and admin API parity

Register one Django permission, courses.registration_count_baseline_manage, for course_operator and site_admin. Declare each adapter in the shared capability registry; Studio and API call the same course services and enforce the same validation, authorization, revision, idempotency, result, and audit policy.

Studio:

  • GET/POST /studio/courses/registration-count-baselines/ — list safe runs or stage one registered source;
  • GET /studio/courses/registration-count-baselines/<uuid>/ — safe run/revision detail;
  • confirmed POST detail actions dry-run, validate, activate, cancel, and rollback;
  • GET /studio/courses/registration-campaigns/<slug>/public-count/ — protected completeness/contribution preview.

Admin API:

  • GET/POST /api/v1/admin/course-registration-count-imports;
  • GET /api/v1/admin/course-registration-count-imports/<uuid>;
  • confirmed POST /api/v1/admin/course-registration-count-imports/<uuid>/{dry-run,validate,activate,cancel,rollback};
  • GET /api/v1/admin/registration-campaigns/<slug>/public-count.

POSTs require Idempotency-Key, explicit confirmation, and a bounded reason code; state transitions also require the current revision (If-Match in the API and an equivalent hidden revision in Studio) and return 409 on stale state. Safe reads are bounded/paginated. Denied or cross-object lookups do not reveal existence. Every response is private/no-store/noindex and exposes only safe IDs, public slugs, integer counts, checksums, states, revisions, timestamps, completeness, and bounded codes—never the source reference/path or registration fields.

Audit stage/validate/activate/cancel/rollback/replacement and denied high-risk attempts with actor, target UUID/public label snapshot, idempotency/correlation ID, prior/new revision/state, safe checksums/counts, reason code, and outcome. Do not audit source locators, row identifiers, registration timestamps, emails/names, free text, account/member fields, consent, or payloads.

Scoped implementation

  1. Add the three course aggregate models, portable migrations, constraints, source adapter, application services, capability declarations, Studio adapters/templates, admin API routes/schemas/OpenAPI, and redacted audit behavior.
  2. Change only the copied registration view’s context seam to call the shared total query; record that source-file overlay with pinned/current checksums and rationale. Do not edit the copied registration template.
  3. Preserve courses_courseregistration in the sensitive-table denylist and all Restore the copied CMP design on course-facing pages #128 exact artifact/import/protected-target checks. The aggregate source registry/import is separate from the frozen development-content artifact.
  4. Update specifications 04, 06, 07, 09, and 10 with this course-specific baseline/native/replacement/privacy/management contract and update adoption integration provenance for the view overlay.
  5. Add focused synthetic model/import/service/view/Studio/admin API/OpenAPI/privacy/browser coverage and a deployment-safe read-only assertion for the copied row. Use only uv and repository Make targets.

Non-goals

  • No raw/production database access during ordinary tests, committed source artifact, PII fixture, or real-source value in code.
  • No CourseRegistration, fake email/account/member/enrollment, shadow registration UI, second template, or hard-coded 472/475/478/1,915 value.
  • No weakening or replacement of Restore the copied CMP design on course-facing pages #128’s sanitized artifact, allowlist, sensitive-table zero check, import receipt, copied course runtime, or current visual provenance.
  • No edit to the copied registration template, Restore the copied CMP design on course-facing pages #128 breadcrumb/accessibility CSS/deployed locator remediation, registration form fields, validation, confirmation, email/Datamailer behavior, campaign lifecycle, route, or SEO contract.
  • No reuse or modification of the event-specific Luma/Eventbrite aggregate models/policies.
  • No full row-level registration/account/profile/consent migration, deletion/retention decision, source disposal, AWS transport, deployment, or production mutation.
  • No public totals API, attendee list/export/search, campaign analytics dashboard, per-country/role/company breakdown, unique-person claim, or attendance/enrollment inference.

Acceptance criteria

  • Portable course-owned models enforce immutable source/count revisions, exact states, complete-run activation, one campaign/cohort slot, revision-guarded pointer transitions, and retained rollback history.
  • The registered protected-source adapter verifies exact snapshot/schema/policy/manifest identity, derives counts only by exact campaign and recorded cohort, and rejects null/mixed/missing/duplicate/stale mappings or cutoff violations without selecting or retaining PII.
  • Empty target import is atomic; exact replay is a no-write deterministic result; same-identity/different, partial, changed-schema/checksum, crash, and concurrent import/activation fail closed without altering the active set.
  • The public query returns only a complete integer and computes baseline plus exact post-boundary native rows once; other campaign/cohort, pre-boundary, null, and ambiguous rows do not contribute.
  • Synthetic aggregate-to-row replacement atomically reconciles the baseline to preserved pre-boundary rows, keeps the visible total unchanged, prevents simultaneous aggregate/row counting, and supports safe pointer rollback; mismatch/overlap blocks replacement.
  • Missing, incomplete, quarantined, stale-target, stale-source, and failed-replacement states omit the row rather than claim zero; a failed refresh preserves a still-valid prior active value; infrastructure failure is not relabelled as zero.
  • The copied page preserves exact zero-hidden, 1 already registered, many-count, and edition-suffix behavior at desktop/mobile, with invariant registered wording and no template redesign.
  • A successful new website registration increases the rendered complete total by one after commit without changing the baseline, and replay/duplicate form submission does not increment twice.
  • Studio/admin API capability, shared-service, permission/denial, private-cache, pagination, revision/If-Match, idempotency, safe-result, OpenAPI, audit/redaction, and rollback parity covers every listed operation.
  • Database, HTML, accessibility tree, API, OpenAPI examples, logs, audits, metrics, errors, screenshots, test artifacts, and committed fixtures contain no real or source-derived registration row, email/name/account/member/IP, attendee digest/identifier, free text, consent/newsletter value, source locator, or provider payload.
  • review_import.manifest still classifies courses_courseregistration as sensitive; the approved Restore the copied CMP design on course-facing pages #128 public-content artifact and exact empty/match/different protected-target contract remain green and no fabricated row is introduced.
  • Specs/adoption ledgers, focused Django/model/migration/import/security/privacy/Studio/API/view tests, copied CMP characterizations, make test-playwright-core, and applicable lint/format/type/migration/Django/OpenAPI/parity/portability/adoption checks pass.
  • [HUMAN] An authorized operator registers one exact protected CMP database snapshot, verifies its checksum/schema/freeze and aggregate facts against the source registration pages at the same boundary, performs dry-run/replay/activate/rollback in development, and captures only redacted aggregate evidence proving the copied row and zero retained/exposed PII.

Automated and browser scenarios

  1. Synthetic source snapshots: zero/one/many rows; two campaigns/cohorts; null or changed course; one campaign spanning two cohorts; naive/post-cutoff timestamps; missing/extra column/table; changed bytes/schema/manifest; duplicate target mapping; oversized/unreadable/non-regular source.
  2. Lifecycle: empty stage, exact replay, same checksum with different payload, newer snapshot, partial insert, failed validation, concurrent activation, stale revision, cancel, rollback, campaign repoint, source-registry checksum drift, and prior-active fallback.
  3. Totals: zero baseline/native, baseline only, native only when coverage explicitly permits it, baseline plus one/many native rows, wrong campaign/cohort, pre-boundary row, duplicate form replay, incomplete/stale/quarantined/error, and aggregate-to-row replacement/rollback with invariant total.
  4. Studio/API: course operator and site admin allowed; unrelated operator, anonymous user, missing scope, cross-object and stale actor denied; idempotency conflict, If-Match, confirmation, safe errors, OpenAPI and audit redaction are identical.
  5. At approximately 1440×900 and 390×844 in light/dark mode, render the copied registration page with 0, 1, and plural synthetic totals; exercise a successful registration increment, missing/stale state, keyboard/focus, 200% zoom/reflow, local assets, and no JavaScript. Compare the registration content region with the exact pinned CMP template, inspect every screenshot, and assert no overflow, clipping, console/request error, debug page, or protected value in DOM/accessibility/network/history/evidence.

Dependencies and delivery

Merged #30/#128 adoption/runtime work supplies the copied model/view/template and sanitized campaign metadata. This issue does not depend on closure of #128’s remaining owner/deployed gate and must not absorb its four-path breadcrumb/smoke remediation; if that accepted commit lands first, start from the merged result and keep its focused tests green.

#64 governs eventual protected-source retention/disposal and the full PII migration; it is not a prerequisite for building/storing this minimized aggregate. The real protected-source run is the named [HUMAN] gate. The later Course → Cohort/full-registration migration must consume the replacement contract above rather than bypassing it.

Implementation uses synthetic sources only. Because real-source reconciliation requires authorized protected data, commit accepted code with Refs #133, retain/add human, and leave #133 open until the owner/operator gate passes. No pull request is created; follow the independent engineer → tester → PM → local merge/on-call lifecycle.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P0Must-have or release-blockingaccessibilityArea: accessibilityadminArea: adminbugSomething isn't workingcoursesArea: coursesdata-migrationArea: data-migrationfrontendArea: frontendhumanCode complete; manual verification requiredintegrationArea: integrationoperationsArea: operationssecurityArea: securitytestingArea: testing

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions