Review CMP system-evaluation and notification behavior from 598c028f

[alexeygrigorev]

Parent: #148

Context

CMP commit 598c028fcd5cf30d4b1cdcae344bcc2193adc2c9 introduces substantive adopted-platform behavior beyond the current target overlays. The exact review identified a cohesive feature set:

• system-project-evaluation models/exports, API schemas/operations, migration, and consumers;
• explicit peer-review and score notification actions in Studio Courses templates, URLs, views, and characterization tests;
• notification side effects moved out of assignment/scoring paths.

The scheduled drift report 31745422903 is status=blocked, applied=false; no source bytes have been copied or applied. #148 records the per-path checksums and keeps the target-owned overlays intact.

Scope

Review the upstream feature at the immutable source commit and determine which behavior is required for the adopted course platform. Preserve the literal CMP implementation and tests when accepted, while retaining the target-owned Studio Courses namespace, security boundaries, registration aggregate seam, and project-state presentation overlays. Where behavior is not in scope for the website release, record an explicit exclusion and rationale.

If accepted, prepare the minimal target overlay updates and source-pin synchronization as a separate reviewed change; do not hide substantive behavior inside a checksum refresh. Preserve migration history and add any required migration/upgrade/replay evidence.

Acceptance criteria

• A reviewed contract maps every upstream changed file and behavior to an explicit disposition: adopt, defer with rationale, or separate follow-up.
• Course-domain owners confirm whether system evaluation and peer-review/score notification behavior belongs in the current release; unknown policy remains fail closed.
• Adopted behavior is copied literally from the exact source commit where possible, with only recorded target-owned overlays retained. No reimplementation or second course UI is introduced.
• API, Studio Courses routes/templates, notifications, model/migration, and characterization tests preserve the CMP behavior and the website's existing security/identity/Relay boundaries.
• Migration tests cover fresh install, upgrade from the pinned graph, idempotent replay, and SQLite ordinary CI; no production/private data is used.
• Exact CMP sync dry-run becomes conflict-free (ready or no_change) only after every affected overlay is reviewed and the source pin/manifest change is independently tested; no --apply occurs before that gate.
• Independent tester and PM reports cover the changed domain and render-impacting Studio/API surfaces, including desktop/mobile screenshots where applicable.

Non-goals

• No protected production database/export access.
• No automatic CMP merge, GitHub PR, or source-repository mutation.
• No changes to unrelated events, content, SEO, or infrastructure.
• No bypass of Preserve public course signup totals without registration PII #133's human-gated registration-total evidence.

Dependencies

• Review CMP upstream overlay drift at 598c028f #148 exact overlay review and source/report evidence.
• Existing copied CMP adoption contract and Studio Courses rename (Rename the copied cadmin package to Studio Courses without changing behavior #116).
• Registration aggregate overlay remains constrained by Preserve public course signup totals without registration PII #133.
• Relay/email behavior must continue to follow Integrate Relay for website email and retire Datamailer #106/Reconcile normative email specifications to Relay ownership #124 and the approved purpose policy.

Follow _docs/PROCESS.md: PM grooming, isolated engineer worktree, independent tester, PM acceptance, then local merge/push; no pull request.

[alexeygrigorev]

PM grooming — BLOCKED pending course-domain decision

Verdict: BLOCKED; not ready for engineering. The source change is a real product/integration change, not a checksum refresh. No course-domain owner decision approving (or rejecting) these behaviors is recorded, so I am not inventing one. Keep the source pin at 98a235283904b4ef9ad29e196298540756cf1bcc, do not copy source bytes, and do not run --apply.

Evidence boundary

• Review CMP upstream overlay drift at 598c028f #148 reviewed the immutable CMP request 598c028fcd5cf30d4b1cdcae344bcc2193adc2c9, source tree a37115e63382a317730e2c6fe5d8ef7a27ac17be, against pinned tree 22a4a9944f353bce1f7779c52cf3ec0fa56a7f32.
• Scheduled run 31745422903 / artifact 9198777578 is status=blocked, applied=false: 31 allowlisted changes, 6 excluded changes, 18 copy entries, 13 overlay conflicts, and 0 fatal errors. Report SHA-256 is 6bde1087fe5f7b988f6d8290559ea3028502b94ec0a915be8f8fd8d855584a35.
• The exact-SHA dry-run in Review CMP upstream overlay drift at 598c028f #148 has the same result and is fail-closed. Its report SHA-256 is 7f4e13dccbf01122df298b340cb897904a418b25bd8b63b8476724aaa2196b44; no source pin or target bytes were applied.
• The current Review CMP upstream overlay drift at 598c028f #148 review record binds all 13 target overlays, retains their target bytes, and links each separate_issue disposition to this issue: overlay-review-598c028f.json. Review CMP upstream overlay drift at 598c028f #148 remains the evidence/ownership boundary; Review CMP system-evaluation and notification behavior from 598c028f #149 is the product decision and possible implementation follow-up.

The requested source is the series 8146d463 (separate project processing from notifications), 80d67dfa (show eligible peer-review assignments), 894696cf (system-project-evaluation fallback), and 83242da (hide scores until a project is evaluated). In aggregate it changes 37 source paths / 1,981 lines, including:

• persistent SystemProjectEvaluation and SystemEvaluationCriteriaResponse models plus migration courses/migrations/0041_system_project_evaluations.py, scoring aggregation, result feedback, and score-visibility changes;
• staff-token GET/POST compatibility API, OpenAPI schemas/routes, idempotent rubric submission, and submission/project data exposure;
• Studio Courses notification controls/routes for peer-review assignments and score delivery, while removing notification sends from assignment/scoring actions;
• learner-facing visibility of eligible peer-review assignments;
• 13 target-owned overlay conflicts listed and checksum-bound by Review CMP upstream overlay drift at 598c028f #148; 18 non-overlay allowlisted copies; and 6 non-allowlisted metadata/agent/deployment changes that remain excluded.

Normative constraints

This review follows the #145 CMP runbook and the current adoption/specification authorities. The adopted platform is a literal copy with explicit target overlays; changed overlays fail closed until reviewed. The #116 decision keeps the logical namespace Studio → Courses. The #133 course registration aggregate is technically accepted but still has its authorized protected-source [HUMAN] gate open; #149 must not consume, change, or claim that gate.

The course specs already preserve peer review, scoring, results, statistics, and communications, but they do not approve these new system-evaluation, assignment-visibility, score-hiding, or explicit-send semantics. Specs 04/06/07/09/10 require course/cohort isolation, capability/object authorization, CSRF, revision/idempotency controls, safe errors/redaction, migration rehearsal, and desktop/mobile verification. #106 and #124 make Relay the sole new website sender: business mutation plus one logical delivery intent commit first, a durable job contacts Relay after commit, and new website code must not send through Datamailer or SES. Any unresolved purpose/sender policy remains fail closed.

Required owner decision before engineering

A course-domain owner must explicitly answer, in an issue comment or linked decision record:

1. System evaluations: Are machine/instructor evaluations a supported release behavior? Who may create them (named staff capability, trusted service, or both)? Must every criterion be answered? Are feedback, rubric answers, score aggregation, replay, correction, and deletion/retention semantics approved?
2. Peer-review assignment visibility: Should eligible learners see assignments on course pages, under which cohort/project/privacy conditions, and what must remain hidden until assignment/submission state permits it?
3. Score visibility: Is hiding scores/results until any peer or system evaluation intentional? Define learner, staff, API, and incomplete/error behavior; unknown or stale evaluation state must not become a misleading score.
4. Notifications: Are separate explicit peer-review and score notification actions desired? Define who may trigger them, confirmation and repeat behavior, audience/purpose/template, Relay delivery status, and whether a failed send is retryable without duplicate delivery.

No “owner approved” conclusion is inferred from the upstream commit, #133, #145, #148, #106, or #124. If the owner defers or rejects a family, #149 records that disposition and leaves the corresponding overlays/source changes unapplied.

Bounded implementation scope after a decision

• Produce a machine-readable contract under _docs/adoption/course-platform/ mapping all 37 changed source paths (31 allowlisted, 6 excluded) and behavior families to adopt, defer/exclude, or a separately tracked issue. Preserve Review CMP upstream overlay drift at 598c028f #148’s exact source/tree/report and all 13 overlay checksums; no silent copy.
• For adopted behavior, copy non-overlaid CMP bytes literally where compatible, retain studio_courses and other target-owned overlays, and use shared application services consumed by public views, Studio, API, and jobs. Do not reimplement a second course workflow or add model exports without their owning implementation.
• For system evaluations, define portable migration/upgrade/replay behavior, uniqueness/idempotency, complete-rubric validation, scoring/leaderboard effects, actor authorization, result visibility, redacted audit, and fail-closed error states. Preserve course/cohort isolation and migration history.
• For notification actions, separate the business transition from sending; route Studio/API mutations through the owning service and capability registry. Commit the business change and one logical intent atomically, enqueue a durable post-commit Relay job, use stable idempotency, and expose retry/ambiguous/error states without direct Datamailer/SES calls.
• For learner-facing assignment/score changes, preserve privacy/no-store/noindex, safe incomplete/error behavior, accessibility, existing route compatibility, and exact target-owned visual language.
• Treat source compatibility API paths as an explicit contract decision; any new management capability must also satisfy the /api/v1/admin/ parity, OpenAPI, permission, revision, idempotency, and audit requirements rather than relying only on a staff-token shortcut.

Non-goals

• No CMP source checkout mutation, source-repository change, automatic synchronization, --apply, source-pin movement, commit, push, or pull request in this issue.
• No protected/production database, registration export, PII, AWS/Relay secret, provider request, or real email. Do not bypass Preserve public course signup totals without registration PII #133’s human gate.
• No redesign of Course → Cohort, registration aggregates, unrelated events/content/SEO/infrastructure, or the accepted Rename the copied cadmin package to Studio Courses without changing behavior #116 namespace decision.
• No direct SES or new Datamailer sends, dual-sender fallback, synchronous network call inside a business transaction, or provider-accepted-as-delivered assumption.
• No second course UI, unscoped staff endpoint, learner/attendee export, or weakening of existing security/privacy boundaries.

Acceptance criteria (after unblocking)

• Course-domain owner decision explicitly records adopt/defer/reject for system evaluations, assignment visibility, score hiding, and notification actions; unknown policy remains fail closed.
• The reviewed contract maps all 37 changed source paths, the 13 Review CMP upstream overlay drift at 598c028f #148 overlays, exact source commit/tree and report identities, target checksums, rationale, reviewer, and follow-up issue for every deferred/separate behavior.
• No behavior is adopted without its decision. Adopted bytes remain literal where possible, preserve the studio_courses/target overlays and Preserve public course signup totals without registration PII #133 registration seam, and introduce no second workflow or silent migration rewrite.
• System-evaluation models/services/migration/API contract (if adopted) proves exact authorization, complete rubric validation, idempotent create/replay/conflict, transactional rescore/leaderboard effects, cohort isolation, redacted audit, safe errors, and fresh/upgrade/SQLite migration coverage.
• Peer-review assignment visibility and score-hiding semantics (if adopted) have explicit learner/staff/API/privacy/error contracts, existing compatibility-route coverage, and no accidental score or submission disclosure.
• Notification actions (if adopted) have explicit Studio/API parity, CSRF/capability/object checks, confirmation, stable idempotency, one logical intent, durable after-commit Relay delivery, retry/ambiguity handling, redacted status/audit, and no Datamailer/SES call.
• OpenAPI, capability/management parity, focused unit/Django/API/privacy tests, copied CMP characterization tests, and migration/portability checks pass; no protected or source-derived private data appears in code, fixtures, logs, screenshots, or artifacts.
• Affected learner and Studio surfaces pass desktop/mobile browser/accessibility checks with inspected screenshots; unchanged documentation/manifest-only review evidence remains screenshot not_applicable.
• Adoption verification and migration drift checks pass. A later source-pin/apply issue, after all adopted overlays are implemented and independently tested, must produce exact-SHA dry-run ready/no_change with zero conflicts/fatal errors before any --apply.
• Engineer, independent tester, and PM reports include exact base/head, plan/report digests, command counts, render evidence, and protected-data/apply boundaries under _docs/PROCESS.md.

Dependencies and unblock path

• Hard blocker: explicit course-domain owner decision above; none is currently recorded.
• Review CMP upstream overlay drift at 598c028f #148 review record and exact report evidence; its 13 conflicts remain blocked until this decision is implemented or explicitly deferred.
• Add controlled upstream synchronization from CMP main #145 adoption verifier/runbook and clean immutable CMP source checkout.
• Rename the copied cadmin package to Studio Courses without changing behavior #116 target namespace/compatibility overlay ownership.
• Preserve public course signup totals without registration PII #133 accepted aggregate contract plus its still-open [HUMAN] protected-source gate; no protected data is needed for Review CMP system-evaluation and notification behavior from 598c028f #149.
• Integrate Relay for website email and retire Datamailer #106/Reconcile normative email specifications to Relay ownership #124 Relay ownership and after-commit delivery contract; purpose/sender decisions that remain unresolved must be linked and fail closed.

Once the owner decision is recorded, PM can revise this comment to READY FOR ENGINEERING with the selected behavior families, exact implementation boundary, and any split follow-up issues. Until then, do not assign an engineer or move the CMP pin.