Purge migrated operational knowledge from public git history

[alexeygrigorev]

Remove migrated content/** from public Git history

Status: blocked — awaiting a coordinated HUMAN maintenance window, frozen refs/work, verified private copy, encrypted backup, deploy suppression, rehearsal acceptance, and separate live approval
Tags: docs, migration, process-docs, testing, data, P1, human
Depends on: #166 and #182 (complete); every active public-repo change committed and pushed or explicitly preserved/rejected before the freeze
Blocks: None
Next owner: named HUMAN repository administrator and independent verifier prepare the freeze packet and disposable-mirror rehearsal; no live ref mutation until Tester, PM, and a separate HUMAN go/no-go approve it

Outcome

Remove historical content/** from every published branch and tag in public DataTalksClub/dataops. The general corpus has already moved to the private dataops-knowledge repository; this issue removes the old public Git reachability without changing the accepted application tree.

This is repository hygiene, not guaranteed un-publication. Existing forks, clones, caches, logs, artifacts, archives, and third-party copies may retain previously public objects. A history rewrite does not revoke credentials.

Exact scope

The only path filter authorized by this issue is the repository-root directory:

• content/** across all history

The filter must use exact path semantics for root content/; do not use a broad content glob or match same-named nested directories.

Explicitly excluded:

• Move Podcast Assistant operational knowledge behind the private boundary #144's 118 Podcast operational-knowledge paths. They are deliberately deferred and remain owned by Move Podcast Assistant operational knowledge behind the private boundary #144's classification, private-copy, HUMAN deletion, source-cleanup, test, and acceptance gates. Their paths/blobs must not be added to this rewrite.
• docs/archive/**, templates/**, Podcast paths, source code, tests, workflows, infrastructure, source repositories, private-repository content, and every other path outside root content/**.
• Editing application files, migrating/copying content, deploying the application, changing AWS, creating reusable one-off scripts/tests, or rewriting a shared checkout/worktree.

Any discovered historical alias outside root content/** is reported as out of scope; it does not silently broaden the filter. Completeness means removal of content/**, not all operational knowledge.

Completed prerequisites

• Replace the transitional Tasks table with the final cardId schema #166 and Gate canonical Card lifecycle on the empty production Cards table #182 are complete; the application rollout no longer depends on the old commit IDs.
• The general content/** corpus was moved to the private knowledge repository. Freeze-time HUMAN verification of the exact accepted private revision and backup remains mandatory before destructive execution.
• Move Podcast Assistant operational knowledge behind the private boundary #144 is intentionally deferred from this rewrite.

Phase 1 — freeze, private-copy proof, and recovery

• [HUMAN] Name the repository administrator, independent verifier, security/private-content owner, rollback decision-maker, maintenance window, and contributor communication channel.
• [HUMAN] The authorized private owner confirms, without exposing content or private paths publicly, that the migrated general corpus exists at an immutable accepted private revision and passes the already-approved coverage/integrity and retention/backup checks. Any mismatch aborts.
• Freeze writes. Every active change is committed and pushed through its lifecycle, or explicitly rejected/preserved privately. All participating worktrees are clean and idle; no agent, bot, integration, tag/release operation, or Actions run may move refs during the window.
• Regenerate the complete published head/tag/ref manifest and repository settings immediately inside the freeze. Record exact old object IDs and the pre-rewrite main tree ID. Historical snapshots in comments are not execution input.
• Disable the application deploy workflow before any rewritten ref is pushed, record its prior state, and drain/cancel active runs. Keep deployment suppressed until remote verification completes; the rewrite must not deploy the application.
• Create an encrypted, access-restricted, immutable mirror/bundle of every pre-rewrite published ref, with checksum, restore proof, retention owner, and expiry decision. It contains public-sensitive history and must never be stored in this public repository or issue.
• [HUMAN] Security records credential/exposure disposition privately. Any credential that may have been public is revoked/rotated independently; rewriting history is not credential remediation.

Any ref, setting, workflow, worktree, private-copy, or backup mismatch aborts before mutation.

Phase 2 — disposable-mirror rehearsal

• From the frozen remote, create a new disposable private mirror outside every shared checkout/worktree. Pin and record the git-filter-repo version and exact command privately.
• Analyze and then rewrite only root content/**, using sensitive-data-removal/invert-path mode. Do not use --partial, broad globs, an unreviewed --mirror push, or any option retaining replacement refs/old objects.
• Save the old-to-new published-ref/commit map privately. Confirm every published branch/tag has exactly one reviewed disposition.
• Confirm the rewritten main tip has exactly the same tree ID as frozen main (root content/ is already absent from the live accepted tree). Confirm no non-content/** path changes between corresponding tips.
• In a normal checkout of the candidate, run the then-current full public verification. No private checkout, token, content, reusable migration script, or new one-off test is introduced.
• Independent Tester verifies the candidate, ref map, scope boundary, backup/restore proof, and safe removal evidence. PM accepts the operator packet.
• [HUMAN] After Tester PASS and PM acceptance, the administrator gives a separate explicit go/no-go for live ref replacement. Issue grooming or rehearsal approval is not live approval.

Discard the rehearsal mirror. The accepted live candidate must be reproduced from a new frozen mirror using the pinned inputs.

Phase 3 — live guarded replacement

• Immediately re-fetch remote refs and settings. Any difference from the signed freeze manifest aborts.
• Reproduce and verify the accepted candidate from a fresh disposable mirror with no implicit/unreviewed destination.
• Replace only the enumerated published heads/tags, each with an explicit force-with-lease expectation for its recorded old object ID. A lease failure aborts; do not retry by weakening the lease and do not use an unreviewed blanket mirror push.
• Keep Actions/deploy disabled during replacement and remote verification. Do not guess at or force-update GitHub hidden refs.
• If server-retained pull-request refs, cached views, or unreachable objects require removal, the named administrator coordinates with GitHub Support using private evidence.

Phase 4 — independent fresh-clone verification

Use a third brand-new clone from GitHub, not either rewrite mirror.

• Published branch/tag names and tips exactly match the approved post-rewrite map.
• Post-rewrite main tree ID equals the frozen pre-rewrite tree ID, and non-content/** trees are unchanged.
• Across every reachable published ref, root content/** has no history, no object-name entries, and none of the privately recorded pre-rewrite content/** blob IDs remain reachable/present.
• git fsck --full --no-reflogs and the approved private secret scan pass. Safe aggregate results only are posted publicly; no filenames, object contents, private locations, or secret-shaped values are exposed.
• Retained Actions artifacts/caches, releases, Pages/packages, and GitHub surfaces are reviewed privately. Deletion requires separate HUMAN approval; GitHub Support completion is verified where applicable.
• Full public verification is green in the fresh clone. On-Call confirms no application deployment ran, production remained unchanged, and remote fetch/push health is normal.
• Restore recorded repository restrictions/integrations and validation workflows. Restore the deploy workflow only after On-Call confirms the rewrite did not deploy and the production/application tree is unchanged.
• [HUMAN] Notify contributors to quarantine/delete stale clones and reclone. Old branches must not be pushed because they can republish removed objects.
• PM performs final acceptance; HUMAN closes the issue.

Pack size and commit-count changes are diagnostics, not removal proof.

Rollback

Before remote mutation, failure means discard the candidate and end the freeze.

After mutation, keep writes and deploy disabled. Prefer correcting the accepted candidate while preserving removal. Only the named HUMAN rollback decision-maker may restore exact old refs from the encrypted backup, using the same explicit leases and freeze controls, after Security accepts that rollback deliberately republishes the removed history. Verify restored refs/tree/settings and schedule a new window.

Test scenarios

Moving repository

Given a changed ref/setting, active writer/run, dirty participating worktree, failed lease, or private-copy/backup mismatch
When preflight or the live re-fetch runs
Then abort before GitHub mutation and do not weaken the gate.

Exact content-only candidate

Given frozen refs and exact root content/** scope
When the disposable mirror is rewritten
Then content/** history/objects are unreachable, all other corresponding path trees are unchanged, published refs have mapped successors, and full public checks pass.

Safe live replacement

Given Tester PASS, PM acceptance, explicit HUMAN live approval, deploy suppression, and unchanged leases
When enumerated refs are replaced
Then a fresh GitHub clone proves removal and tree parity, and no application deployment occurs.

Deferred Podcast cleanup

Given #144 remains open
When #175 completes
Then its 118 Podcast paths remain unchanged and #175 claims only root content/** removal.

Lifecycle

• PM grooming updated for explicit content/**-only intent
• HUMAN/operator freeze and private-copy/backup packet
• Disposable-mirror rehearsal
• Tester verification
• PM candidate acceptance
• [HUMAN] separate live go/no-go
• Guarded live rewrite
• On-Call fresh-clone/no-deploy verification
• PM final acceptance
• [HUMAN] close issue

[alexeygrigorev]

PM GROOMING PASS — blocked on coordinated HUMAN freeze

#175 is now implementation-ready as a destructive-operations runbook, but it is intentionally unsafe to execute in the current repository state.

Current evidence: local main is seven commits ahead of origin/main; the shared checkout is dirty; 94 local branches, 19 worktrees, one stash, and active issue work exist; and the repaired #166 A/B/C chain is local-only. The issue makes preservation of those commits and every other accepted workstream a hard prerequisite.

The groomed plan now defines:

• the exact initial filter allowlist and explicit exclusions;
• full local/remote ref and worktree inventory;
• encrypted immutable backup plus old-to-new ref/commit mapping;
• credential rotation as a separate security gate;
• coordinated write/Actions/deploy freeze;
• two disposable-mirror rehearsals before any remote mutation;
• independent Tester and PM review before a separate live HUMAN approval;
• guarded per-ref force-with-lease replacement instead of an unreviewed mirror push;
• fresh-clone path/blob/object verification, GitHub Support handling, Actions/artifact/cache and release/tag checks;
• mandatory contributor recloning, with exceptional patch replay replacing blind old-history rebase;
• rollback that explicitly records the risk of republishing the old corpus.

Next owner: the named repository administrator, but only after the issue's resume condition is met, for a non-mutating rehearsal in a disposable private mirror.

No history was rewritten, no ref or repository file was changed, no workflow was disabled, no credential was rotated, and nothing was force-pushed during grooming.

[alexeygrigorev]

PM lifecycle reconciliation — snapshot refreshed; freeze-time regeneration is mandatory

The destructive plan remains correctly blocked. Read-only metadata confirms the earlier snapshot is already obsolete:

• origin/main is still 7940763 with one published head and zero published tags, but local main is now e54b73f, 12 commits ahead, with eight tracked modifications.
• Local state is now 95 branches, 20 worktrees (three detached), three stashes, and six dirty worktrees. These are observations only; no names/paths from dirty state are an execution manifest.
• The safe Replace the transitional Tasks table with the final cardId schema #166 sequence is still repaired A 55b2d00 → B f32af76 → C b6b1ce9. Current integration 0a9322c descends from repaired C and the accepted Card/writer preparation, but does not yet contain Keep one-off import verification out of ongoing backend CI #187 50df377 or Restore behavior-based browser coverage and fix canonical UI accessibility races #180 work. Local main contains the revoked original A/B/C line and is not a safe rollout push.
• Purge migrated operational knowledge from public git history #175 must occur after Replace the transitional Tasks table with the final cardId schema #166/Gate canonical Card lifecycle on the empty production Cards table #182 A/B/C/D/cleanup and restored ordinary deployment. Rewriting earlier would change the exact SHAs and ancestry that the reviewed rollout gates require.
• Move Podcast Assistant operational knowledge behind the private boundary #144 is now a pre-freeze dependency for a single comprehensive rewrite. Its 118 Podcast paths are not automatically authorized for filtering; after private classification/copy and exact public cleanup approval, their historical paths/aliases/blobs must be separately added and reviewed. Explicit deferral must be recorded if they remain outside Purge migrated operational knowledge from public git history #175.

The issue now requires complete inventories to be regenerated from scratch immediately after the coordinated write freeze, signed by an independent verifier, and regenerated/compared again immediately before live ref replacement. This includes remote/GitHub surfaces, every local branch/worktree/stash/reflog-only commit and disposition, the accepted preservation graph, exact removed paths/aliases/shared blobs, settings/workflows, backup, and credential/exposure decisions. Any delta aborts the window.

The old 2,147-path count remains historical context for b321b8c, not an authorized total. Current comments, SHAs, counts, branch names, and worktree state may not be patched forward into the operator packet.

No history/ref/workflow/repository setting was mutated; no sensitive content was inspected; no repository file, AWS/provider/credential, private repository, import/export/restore, or data action occurred.