Gate canonical Task/Card transactions in DynamoDB Local and trim duplicate browser API specs

[alexeygrigorev]

Gate canonical Task/Card transactions in DynamoDB Local and trim duplicate browser API specs

Status: blocked on #166 cleanup — Stage A is Tester/PM accepted and committed locally as ae26ba6; Stage C is Tester/PM accepted and committed locally as bf5b39c; only Stage B implementation remains, followed by combined verification
Tags: enhancement, work-engine, backend, frontend, infra, testing, P1
Depends on: #166 successful repaired rollout through post-success cleanup and restoration of the ordinary steady-state deploy workflow; #180 is closed and no longer blocks
Blocks: None
Next owner: #166 orchestrator/On-Call first; after #166 cleanup, Software Engineer adds only the accepted Stage A command to steady-state deploy checks, then Tester and PM perform Stage B/combined acceptance
Resume condition: #166 Phase A/B/C/#182/D and cleanup have production PASS evidence and the normal push/OIDC deploy workflow is restored without transitional mechanisms; then implement Stage B only from accepted descendant bf5b39c

Scope

Close two test-architecture gaps without adding another broad integration suite:

1. Add one deterministic, deploy-gating DynamoDB Local check for the canonical Task/Card transaction accepted in Complete and archive cards automatically when all tasks are done #168. The check must run the same production TransactWriteItems path that Lambda uses, including Prevent lost Task updates and history with optimistic concurrency #179's Task version/history condition and Complete and archive cards automatically when all tasks are done #168's Card counter/lifecycle/audit writes. The ordinary Dynalite-backed backend and browser suites deliberately use explicit local transaction emulation and cannot provide this proof.
2. Remove the large Playwright api-* CRUD/validation suites after preserving each meaningful behavior in its proper layer. Backend API/domain tests own request validation, persistence, conflicts, and business rules; Playwright owns a small single-origin routing/auth/deployment-shape smoke plus real operator journeys.

This is test/CI work, not a runtime transaction redesign. Do not add an environment-selected production fallback, general transaction framework, migration/import/restore/export coverage, production data access, or an AWS call.

Accepted local checkpoints

• Stage A — ae26ba609b84d3dd38608b6adc74ec7a51179ddd. Tester PASS and PM acceptance cover exactly backend/package.json, backend/scripts/test-task-card-transaction.sh, and backend/tests/task-card-transaction.ddb.ts. Evidence: official loopback/ephemeral DynamoDB Local, production mode with emulation disabled, real forwarded TransactWriteCommand, semantic complete/reopen/race atomicity, 1/1 focused pass, test TypeScript config, shell syntax, diff check, and cleanup.
• Stage C — bf5b39cbfaad3c026caebe5c5ad4bd048b67398c. This descendant contains the accepted Stage A checkpoint and the exact accepted eleven-path browser convergence: seven broad API-spec deletions, independent retained Template-update journeys, two focused backend owners, and one README command correction. Evidence: focused backend owners 71/71, Template-update journeys 2/2, complete remaining browser suite 63/63 with every discovered case accounted for, typecheck, build/runtime boundary, canonical artifact verification, and diff checks.
• Restore behavior-based browser coverage and fix canonical UI accessibility races #180 closed. The 63/63 Stage C result satisfied Restore behavior-based browser coverage and fix canonical UI accessibility races #180's deferred complete-suite criterion. Restore behavior-based browser coverage and fix canonical UI accessibility races #180 is no longer a dependency or blocker.

Both checkpoints are local and unpushed. Acceptance is not shipment, deploy, or On-Call evidence. Current HEAD is Stage C bf5b39c; origin/main remains behind it.

Staged implementation plan

#183 remains open only for the #166-gated permanent deploy step and final combined verification. Stage A and Stage C are frozen accepted local commits; do not reopen or fold them into Stage B.

Stage A — isolated transaction harness (accepted and committed locally as ae26ba6)

• Implement only the focused DynamoDB Local Task/Card lifecycle test, its isolated runner, and an explicit backend package command.
• Bind the assertions to the accepted production behavior in Prevent lost Task updates and history with optimistic concurrency #179 commit 02ee455efad09865ecdbd578e940c59e0bf7dc36 and Complete and archive cards automatically when all tasks are done #168 commit 41dca9e3748fe627c195f48a7f97ae203b343f60.
• Prove semantic completion/reopen history entries and versions relative to the state returned by the preceding mutation; do not pin an absolute version, history length, transaction-item count, delay, or incidental error copy.
• Do not edit the deploy workflow or any Playwright spec in this stage.
• This stage may receive its own Tester/PM review and a Refs #183 checkpoint commit, but it does not close Gate canonical Task/Card transactions in DynamoDB Local and trim duplicate browser API specs #183.

Stage B — permanent deploy gate (blocked on #166 cleanup)

• After Replace the transitional Tasks table with the final cardId schema #166 has completed its repaired A → B → C → final Gate canonical Card lifecycle on the empty production Cards table #182 preflight → D → cleanup sequence, add the Stage A command to the restored steady-state deploy checks job before packaging.
• Do not add the command to an A/B/C/D transitional workflow, reuse a cutover marker, or retain any Replace the transitional Tasks table with the final cardId schema #166/Gate canonical Card lifecycle on the empty production Cards table #182 phase mechanism.
• Prove with focused workflow tests that the command is in checks, precedes the single build/package path, and needs no AWS credentials.

Stage C — browser convergence (accepted and committed locally as bf5b39c)

• Start only from accepted Restore behavior-based browser coverage and fix canonical UI accessibility races #180 checkpoint 4b5c18bb52d954a92d4ae4331ebdf91d738b4ca1 (parent e54b73f9a06fdb15b014bc0a1aa6268c714a0df4). It is local and unpushed; acceptance is not shipment evidence.
• The frozen debt boundary is exactly eight files:
  1. backend/e2e/api-cards.spec.js
  2. backend/e2e/api-cron.spec.js
  3. backend/e2e/api-files.spec.js
  4. backend/e2e/api-instantiation.spec.js
  5. backend/e2e/api-recurring.spec.js
  6. backend/e2e/api-tasks.spec.js
  7. backend/e2e/api-users.spec.js
  8. backend/e2e/card-template-updates.spec.js
• For the seven broad api-* files, inventory every meaningful assertion, assign exactly one continuing backend/compact-smoke/real-UI owner, add any missing focused owner first, and then delete the broad duplicate file. Do not keep a temporarily green duplicate matrix.
• Preserve card-template-updates.spec.js as real operator coverage, but converge its two journeys to independent synthetic state, the returned current version, and Complete and archive cards automatically when all tasks are done #168's current Template retirement/history semantics. Remove only its pure API duplication and serial/order coupling; do not delete the operator journeys wholesale.
• Missing expectedVersion in legacy browser fixture mutations is evidence of owner drift, not an instruction to mechanically add it to every old request. Prefer already-accepted Prevent lost Task updates and history with optimistic concurrency #179/Complete and archive cards automatically when all tasks are done #168 backend coverage and delete obsolete browser-domain duplicates. Where a retained real UI journey genuinely requires a direct setup mutation, obtain the current returned version at that exact step—never a magic version, bypass, retry, compatibility shim, or weakened conflict contract.
• Preserve Restore behavior-based browser coverage and fix canonical UI accessibility races #180's accessibility/product fixes, diagnostics, per-test isolation, Knowledge bridge, and behavior journeys. Do not edit Replace frontend source greps with runtime behavior coverage #184 capability-evidence/source-grep plumbing.
• Keep validate-backend-e2e.yml independent and non-deploy-blocking, including failure artifact upload.
• The complete independent browser run accounted for every discovered case and passed 63/63 at accepted Stage C commit bf5b39c; the evidence was cross-posted and Restore behavior-based browser coverage and fix canonical UI accessibility races #180 is closed.

Stage D — combined verification and closeout

Stage D closes #183 only after #166 unblocks Stage B, the permanent steady-state deploy gate is Tester/PM accepted and committed, and combined verification passes on the descendant integration. #180 is already closed from the accepted Stage C 63/63 cross-post. No local checkpoint claims shipment.

• Run the full verification listed below only after Stages B and C are integrated.
• Tester records observed discovery/results without encoding counts, timings, byte values, exact incidental copy, source-text matches, or screenshot pixels as product assertions.

Production transaction gate

Implement the smallest focused harness: one test file, one runner, and one deploy-check step. It must:

• start an isolated official DynamoDB Local process/container on a loopback ephemeral port, create only the throwaway tables needed by the accepted Task/Card lifecycle, and always clean up its process/container and tables;
• use synthetic records and final accepted table/key shapes;
• set the local endpoint and fake local AWS credentials while running application transaction code in production mode;
• leave usesLocalTransactionEmulation() false and prove that an actual TransactWriteCommand request is accepted by DynamoDB Local; merely inspecting a mocked command or exercising the local emulator is not sufficient;
• use observable bounded readiness and process-exit diagnostics, not a blind fixed delay;
• exercise one complete lifecycle journey: attached open Task → final Task completion/Card archive → Task reopen/Card reactivation;
• prove the Task version/history mutation, Card version/counters/lifecycle fields, and public-safe lifecycle audit are committed together;
• force a same-version conditional race/failure and prove the loser leaves no partial Task, Card, history, or audit write. Do not pin an implementation-specific command count.

Run this focused command in the deterministic checks job of .github/workflows/deploy-dataops-v1.yml before packaging/deployment. It is a backend correctness gate. Do not add Playwright to that workflow. The runner may share safe container-readiness/cleanup helpers with existing focused DynamoDB Local scripts only when doing so stays smaller and does not turn unrelated transaction suites into ongoing CI.

Browser coverage convergence

Inventory every assertion in:

• backend/e2e/api-cards.spec.js
• backend/e2e/api-cron.spec.js
• backend/e2e/api-files.spec.js
• backend/e2e/api-instantiation.spec.js
• backend/e2e/api-recurring.spec.js
• backend/e2e/api-tasks.spec.js
• backend/e2e/api-users.spec.js

For each behavior, identify exactly one continuing owner:

Behavior	Continuing owner
request validation, CRUD, domain invariants, DynamoDB conditions, concurrency/conflict bodies	focused backend/tests/*.test.ts API/domain tests
operator-visible navigation, state, mutation result, recovery, focus, accessibility	behavior-based Playwright journey
single-origin dispatch, authenticated/unauthenticated boundary, JSON-vs-SPA routing, static/packaged deployment shape	a small Playwright routing/auth/deployment smoke
test setup and cleanup	fixture helpers; not duplicated product assertions

Add missing focused backend coverage before deleting its browser duplicate. Move any genuine UI journey out of an api-* file before deleting that file. Fixture API calls may remain inside UI journeys, but Playwright must not recreate full CRUD/validation matrices with its request fixture.

Delete the seven broad api-* specs above once the inventory is satisfied. Keep a compact behavior-based smoke that proves, through the real local HTTP boundary:

• the health/API namespace is dispatched as API JSON rather than the SPA;
• an unknown API route does not fall through to frontend HTML;
• the existing browser-session/auth tests still prove protected single-origin access and safe auth failure;
• the canonical frontend entry/static module graph loads from the same server shape.

Do not duplicate the product/accessibility repairs, browser diagnostics, fixture isolation, or serial-failure cleanup owned by #180. #183 consumes #180's accepted suite shape and then removes redundant API-contract cases while preserving its meaningful UI journeys.

Acceptance Criteria

• A focused DynamoDB Local runner executes accepted Complete and archive cards automatically when all tasks are done #168 Task/Card lifecycle code with NODE_ENV=production, the explicit local transaction emulator disabled, and at least one real successful TransactWriteItems request.
• Final completion and reopen prove atomic Task version/history, Card version/counters/lifecycle metadata, and public-safe lifecycle audit behavior against DynamoDB Local. Assertions identify the semantic history entries and monotonic versions relative to returned state; they do not pin absolute versions or history lengths.
• A same-version race or failed transaction proves all-or-nothing behavior: the loser cannot partially change Task, Card, history, counters, completion state, or audit, and the accepted conflict contract is preserved.
• The harness uses isolated synthetic local tables/resources, observable bounded readiness, useful process/container diagnostics, and reliable cleanup on pass, assertion failure, startup failure, and interruption.
• The focused transaction command is an ordinary deterministic step in the deploy workflow's checks job and fails that job on regression; no AWS credentials, production resource, import, migration, restore, export, seed, or raw operational data is used.
• No runtime fallback, test-mode branch selected by an environment variable, dual transaction implementation, retry loop, or application-owned table creation is added. Table creation remains test-harness-only.
• Every meaningful behavior formerly in the seven broad Playwright api-* specs has one documented continuing owner; any missing backend/API or UI coverage is added before the duplicate is removed.
• The seven broad Playwright api-* specs are deleted. Remaining Playwright API requests are limited to fixture setup/cleanup, compact route/auth/deployment smoke, or actions inside real operator journeys.
• Meaningful Card/Task, recurring, files, cron/notification, Template instantiation, and user-role behavior remains covered in backend tests and, where operator-visible, behavior-based browser journeys. Coverage is not replaced by exact test-count assertions.
• The retained browser smoke asserts routes, response class, auth boundary, visible state, and usable behavior—not incidental copy, DOM concatenation, internal IDs, byte/digest values, source text, or screenshot pixels.
• Browser cases remain independently runnable after Restore behavior-based browser coverage and fix canonical UI accessibility races #180: no file/describe serial failure dependency, cross-test mutation dependency, fixed synchronization sleep, or test-order contract is introduced.
• .github/workflows/validate-backend-e2e.yml remains the independent, non-deploy-blocking full Playwright workflow and retains failure artifact upload. .github/workflows/deploy-dataops-v1.yml contains no Playwright command or dependency.
• Backend unit/API tests, the focused real-transaction gate, frontend unit tests, and the complete independent Playwright suite pass with all discovered tests accounted for; Tester records observed counts in evidence rather than pinning counts in code.

Test Scenarios

Scenario: Production transaction path completes and reopens a Card

Given: DynamoDB Local has synthetic canonical Cards, Tasks, and lifecycle-audit tables; transaction emulation is disabled; an active Card has one open attached Task
When: production application code completes that Task and then deliberately reopens it from the returned current version
Then: DynamoDB Local accepts real transaction requests, each mutation advances Task/Card state atomically, completion/reactivation audits are retained once, and no emulator/fallback path runs

Scenario: A transaction condition loses

Given: two mutations start from the same canonical Task/Card versions
When: they race or one deliberately invalidates a transaction condition
Then: only one valid transaction commits; the loser returns the accepted conflict behavior and a strongly consistent read shows no partial losing history, counter, lifecycle, or audit effects

Scenario: Harness startup or assertion fails

Given: the local service exits early or the focused test fails
When: the runner reports the failure
Then: diagnostics identify readiness/process state without exposing record content, and owned containers/processes/tables are cleaned up without touching unrelated local resources

Scenario: Duplicate API contract is removed safely

Given: a browser API assertion tests validation, CRUD, a domain invariant, or a conflict already owned by backend tests
When: the Playwright duplicate is removed
Then: the corresponding backend test still proves the behavior through the handler/repository boundary, and no operator-visible journey is lost

Scenario: An API spec contains real UI behavior

Given: an api-* file also asserts canonical Card navigation or another operator-visible flow
When: that broad file is deleted
Then: the UI behavior lives in an independent behavior-based journey with stable route/role/state assertions, while its API CRUD setup remains only a fixture concern

Scenario: Single-origin smoke distinguishes API from SPA

Given: the local production-shaped HTTP server serves the portal and API from one origin
When: the smoke requests health, an unknown API path, authenticated portal/API behavior, and the canonical frontend entry
Then: API responses retain API routing/response semantics, unknown API paths never return the SPA shell, auth behavior remains safe, and the frontend module graph loads

Scenario: CI boundaries

Given: focused local verification is green
When: the change reaches main
Then: the deploy checks run the deterministic transaction gate before deployment, the independent browser workflow runs the full Playwright suite and retains failure artifacts, and no Playwright outcome blocks or queues deployment

Verification

Engineer and Tester record exact commands, exit codes, and observed test counts for:

npm run typecheck:backend
npm run test:backend
npm --prefix backend run test:task-card-transaction
npm run test:frontend:unit
npm --prefix backend run test:e2e
uv run --with pytest pytest tests/infra/test_template.py

The focused transaction command name may be adjusted during implementation only if the final package script is equally explicit and the issue body is updated. Tester must inspect the test runner and CI job to confirm production mode, disabled emulation, loopback isolation, bounded readiness, cleanup traps, and no production/AWS/data operation. No UI source change or new screenshot acceptance is expected here; #180 owns screenshot-backed acceptance for the UI journeys this issue preserves.

After push, On-Call records both the transaction step result in the deploy checks and the independent browser workflow result. Do not trigger a manual deploy, touch AWS, or run source-data scripts merely to verify this issue.

Out of Scope

• Changing the canonical Task optimistic-concurrency contract from Prevent lost Task updates and history with optimistic concurrency #179 or the Card aggregate/transaction contract from Complete and archive cards automatically when all tasks are done #168.
• Generalizing the harness across every existing DynamoDB transaction user, or promoting Sponsor CRM, bookkeeping, Calendar, newsletter, conversational, import, migration, restore, export, or one-off data scripts/tests into ongoing CI.
• Removing the explicit Dynalite transaction emulator used by ordinary local tests; this issue proves the production path separately and must not make the emulator runtime-selectable.
• Product/accessibility fixes, broad browser fixture/diagnostic restructuring, retry policy, or serial-suite repair owned by Restore behavior-based browser coverage and fix canonical UI accessibility races #180.
• A new browser UI journey solely to preserve an API-domain assertion, screenshot goldens, exact suite-size pins, broad text assertions, source-text coverage checks, or longer retries/timeouts.
• SAM/application-stack redesign, production data writes, AWS mutation, source-repository edits, or private operational knowledge.

Dependencies

• Accepted source inputs — satisfied. Prevent lost Task updates and history with optimistic concurrency #179 is accepted at 02ee455efad09865ecdbd578e940c59e0bf7dc36; Complete and archive cards automatically when all tasks are done #168 is accepted at 41dca9e3748fe627c195f48a7f97ae203b343f60. Both are ancestors of the current local source. They are immutable subjects of the Stage A gate, not remaining blockers and not reasons for a compatibility path.
• Replace the transitional Tasks table with the final cardId schema #166 steady-state cleanup — blocks Stage B only. No Replace the transitional Tasks table with the final cardId schema #166 production phase has run. The authorized repaired preparation chain is A 55b2d00f22df45d5d55d9a834e4068da28a1d1da → B f32af76898f3f13501798d7b9b12605da2d70974 → C b6b1ce9031412682b9cf660a92876c312b9a584a, followed by the reviewed Phase D artifact and post-success cleanup. Gate canonical Task/Card transactions in DynamoDB Local and trim duplicate browser API specs #183 must not modify the temporary cutover workflow or assume local D preparation is production evidence.
• Gate canonical Card lifecycle on the empty production Cards table #182 is an in-sequence Replace the transitional Tasks table with the final cardId schema #166 gate, not a second storage migration. After successful Replace the transitional Tasks table with the final cardId schema #166 Phase C and while writers remain quiesced, Gate canonical Card lifecycle on the empty production Cards table #182 repeats the sanitized Cards identity/schema/stable-zero-row audit. A pass keeps the existing conforming Cards table and permits Replace the transitional Tasks table with the final cardId schema #166 Phase D to activate accepted Complete and archive cards automatically when all tasks are done #168; a failure stops and re-grooms the rollout. Gate canonical Task/Card transactions in DynamoDB Local and trim duplicate browser API specs #183 must not replace the Cards table, add Cards migration logic, or wait for a separate post-Replace the transitional Tasks table with the final cardId schema #166 table cutover. Stage B begins only after the resulting Replace the transitional Tasks table with the final cardId schema #166 cleanup restores the normal push/OIDC deploy workflow.
• Restore behavior-based browser coverage and fix canonical UI accessibility races #180 — satisfied and closed. Accepted checkpoint 4b5c18b supplied the Stage C baseline; Stage C bf5b39c produced the fully accounted 63/63 suite and Restore behavior-based browser coverage and fix canonical UI accessibility races #180 closed on 2026-08-13. No Restore behavior-based browser coverage and fix canonical UI accessibility races #180 work remains.
• Remaining implementation. Only Stage B remains. It may edit the restored steady-state deploy workflow to invoke the already-accepted test:task-card-transaction command before packaging, plus the minimum focused workflow test. It must not modify the Stage A harness, Stage C browser/backend owners, or any temporary Replace the transitional Tasks table with the final cardId schema #166/Gate canonical Card lifecycle on the empty production Cards table #182 phase workflow.

[alexeygrigorev]

PM grooming complete.

I reviewed the current Dynalite setup, explicit in-process transaction emulation, focused Docker/DynamoDB Local runners, deploy and independent browser workflows, all seven broad Playwright api-* specs, and the accepted/groomed contracts in #179, #168, and #180.

Decision: blocked on #166, accepted #168, and accepted #180. The real transaction gate must test the final Task/Card aggregate, not today's moving/emulated implementation. #166 must first remove the temporary cutover-only deploy workflow state. #180 remains the owner of product/accessibility fixes, browser diagnostics, fixture isolation, and serial-failure cleanup; #183 follows it and removes redundant API-contract cases without rewriting or weakening its UI journeys.

The implementation is deliberately small:

• one production-mode DynamoDB Local Task/Card lifecycle test, one isolated runner, and one deterministic deploy-check step;
• explicit proof that local transaction emulation is off and real TransactWriteItems succeeds;
• completion, reopen, and one losing conditional transaction proving no partial Task/Card/history/audit writes;
• deletion of the seven broad api-* specs only after every meaningful assertion has one backend/API, UI-journey, or compact route/auth/deployment-smoke owner.

Playwright remains independent and non-deploy-blocking. The issue explicitly excludes generalized transaction matrices, runtime fallbacks, AWS/data operations, migration/import/restore/export suites, exact suite-count pins, source-text coverage, fixed synchronization sleeps, and screenshot goldens.

No repository files, tests, AWS resources, or data were changed during grooming.

[alexeygrigorev]

PM dependency re-grooming complete.

Decision: #183 remains blocked as a whole, with Stage A explicitly implementation-ready now.

Read-only findings:

• Prevent lost Task updates and history with optimistic concurrency #179 commit 02ee455efad09865ecdbd578e940c59e0bf7dc36 and Complete and archive cards automatically when all tasks are done #168 commit 41dca9e3748fe627c195f48a7f97ae203b343f60 are accepted and present in current ancestry. They are now fixed gate inputs, not open dependencies.
• The production Task/Card path is in backend/src/db/taskCardLifecycle.ts: explicit local emulation is selected only through the in-process helper, and the non-emulated branch sends a real TransactWriteCommand. There is no focused Task/Card DynamoDB Local package command yet.
• The seven broad browser API specs still exist and total 2,990 lines. They mix backend-domain CRUD/validation with a small amount of real navigation/routing behavior, so deletion must follow a behavior-owner inventory against the accepted Restore behavior-based browser coverage and fix canonical UI accessibility races #180 suite.
• Restore behavior-based browser coverage and fix canonical UI accessibility races #180 is pending, not accepted. It still owns the genuine accessibility/product fixes, browser races, diagnostics, fixture isolation, and serial-failure repair.
• No Replace the transitional Tasks table with the final cardId schema #166 production phase has run. The repaired A/B/C identities are now exact in the issue. Gate canonical Card lifecycle on the empty production Cards table #182 is correctly represented as the final Cards preflight inside Replace the transitional Tasks table with the final cardId schema #166, not as a request for a second table replacement or migration. The permanent gate may enter the deploy workflow only after Phase D succeeds and Replace the transitional Tasks table with the final cardId schema #166 cleanup restores the ordinary steady-state push/OIDC workflow.

The issue now has four bounded stages:

1. isolated production-path DynamoDB Local harness/runner/package command — locally unblocked;
2. permanent steady-state deploy check — blocked on Replace the transitional Tasks table with the final cardId schema #166 cleanup, including the in-sequence Gate canonical Card lifecycle on the empty production Cards table #182 preflight;
3. browser coverage convergence — blocked on Restore behavior-based browser coverage and fix canonical UI accessibility races #180 Tester/PM acceptance;
4. combined verification and closeout.

Stage A explicitly forbids workflow and Playwright edits and may be independently reviewed/committed with Refs #183. Its assertions are behavior-based: semantic history entries and versions relative to returned state, with no absolute history arithmetic, command-count pin, fixed sleep, byte/text pin, or runtime fallback.

Labels remain appropriate and needs grooming is absent. No repository file, test, AWS resource, workflow, import/export/restore path, or data was touched during this re-grooming.