DOC-6951 Date each redirect from git history, and say why deletions are not published - #3775
Merged
Conversation
Contributor
Contributor
🧠 Redis MemoryFound 6 related items from repository history:
Memory updated at dd114ce |
Contributor
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 0d15581. Configure here.
…re not published Item D3, half of it built and half of it deliberately not. The built half is the dates. The map is rendered from .Aliases, which is everything Hugo knows, and Hugo cannot say when a page moved because frontmatter does not record it. git does. build/generate_page_moves.py writes those dates into data/page-moves.json before the build and the template attaches a `moved_on` to each redirect it can date: 570 of 1,061 entries. The rest are vanity or legacy paths that were never a page's location, so there is no date to give -- which the docs now say explicitly, because an absent date could otherwise read as an undated move. Generated rather than committed, gitignored alongside data/examples.json and the other derived files here, so a snapshot of git history cannot go stale. Every lookup tolerates the file being missing, so a bare `hugo` without `make` still builds and simply publishes no dates; verified both ways. The half not built is deleted pages, and the measurement is the reason. git cannot reliably tell a deletion from a move it failed to detect. Of 195 apparent deletions in this history, 83 have a same-named page elsewhere today -- /develop/reference/cluster-spec against operate/oss_and_stack/reference/cluster-spec.md, and so on -- so they almost certainly moved by a delete-plus-add below the rename similarity threshold. The other 112 are no safer, because a page can be renamed and relocated at once and no name-matching heuristic sees that. Publishing those as deleted would tell a consumer to discard a citation that still resolves, which is worse than telling them nothing. So the docs now state that a URL missing from the map is not necessarily gone, and that we deliberately do not claim deletions. That is a better answer than a list that is 40% wrong under the first check anyone would run against it. Verified on a full build with a production baseURL: 1,061 redirects, 27 ambiguous, 10 shadowed, 570 dated, tombstone coverage still 1,061 from 1,061, and the same ten pre-existing REF_NOT_FOUND warnings that main already emits. Learned: the deletions half looked like the easy other half of this item and was the part that could not be done honestly -- 83 of 195 apparent deletions had a live successor, so shipping the list would have instructed consumers to drop working citations Constraint: the map never claims a page was deleted, because git cannot distinguish a deletion from an undetected move, and a wrong deletion is worse for a consumer than no entry at all Constraint: data/page-moves.json is generated before each build and gitignored, so the dates cannot drift from the history they come from Ticket: DOC-6951 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…th no dates The dates would have been empty in production. generate_page_moves.py reads git rename records, and the only job that runs `make ci` checks out with actions/checkout's default fetch-depth of 1. A shallow clone has no rename records, so the scan finds nothing, reports zero moves and exits perfectly happily -- the map would have shipped without a single date while every local build showed hundreds. Verified by running the generator inside a --depth 1 clone: 0 dates, no error. Two changes. The latest build now checks out with fetch-depth: 0, and only that job, since the versioned matrix builds run bare `hugo` and never generate the map. And the generator now detects a shallow clone, warns as an Actions annotation, and records `shallow_clone` in the output so nobody has to guess whether a file with no dates means a corpus without moves or a clone that could not see them. This is the third time in this ticket that correct code was defeated by a default in its surroundings: checkout depth in the alias workflow, the baseURL prefix in the map template, and the Actions shell's errexit. All three were invisible locally and none would have failed a test. The pattern is worth more than any of the individual fixes: verify under the conditions CI creates, not the ones a laptop happens to have. Learned: a shallow clone makes any git-history scan return an empty answer rather than an error, so the absence of data has to be distinguishable from an absence of history Constraint: the latest build needs fetch-depth 0 because make ci reads git rename records; the versioned builds do not, and should not pay for it Ticket: DOC-6951 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
classify() ran on every build and nothing read what it set. It decides whether each move is already aliased, shadowed by a live page, contested or aimed at a draft, and that costs a scan of every published URL plus the frontmatter of every file declaring an alias. Dating needs only old_url and date. Removing the call leaves the output byte-identical -- same 615 dates, same payload -- and takes the generator from 2.78s to 1.04s. Small in absolute terms, but it runs on every build of the site, and work that nothing consumes is worth none of it. It was there because this generator was written by importing the scanner's pipeline wholesale, and find_moves plus classify is how every other caller uses it. Reusing a pipeline is right; reusing the parts of it you do not need is just habit. Learned: importing a pipeline wholesale carried a scan whose results were never read -- reuse the function, not the sequence of calls around it Ticket: DOC-6951 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
andy-stark-redis
force-pushed
the
DOC-6951-page-moves
branch
from
August 10, 2026 13:43
1c4b848 to
dd114ce
Compare
Contributor
Author
|
Thanks @dwdougherty ! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Item D3 of DOC-6951 — the last item. Half built, half deliberately not, and the half I didn't build is the more interesting result.
Built: a date on every redirect we can date
The map is rendered from
.Aliases, which is everything Hugo knows — and Hugo can't say when a page moved, because frontmatter doesn't record it. git does.build/generate_page_moves.pywrites those dates intodata/page-moves.jsonbefore the build, and the template attachesmoved_onto each entry it can date: 570 of 1,061.{"from": "/develop/ai/langcache", "to": "https://redis.io/docs/latest/develop/ai/context-engine/langcache/", "moved_on": "2026-05-11"}The value for a consumer is separating "this redirect is two years old, I've surely seen it" from "this appeared last week, my index is stale". Dates span 2024-02-19 to 2026-08-05.
The other 491 entries are vanity or legacy paths that were never a page's location, so there's no date to give. The docs say that explicitly, because an absent date would otherwise read as an undated move.
Generated, not committed — gitignored alongside
data/examples.jsonand the other derived files here, so a snapshot of git history can't go stale. Every lookup tolerates the file being absent, so a barehugowithoutmakestill builds and simply publishes no dates. Verified both ways.Not built: deleted pages, and why
This was the other half of the item, and it can't be done honestly. git cannot reliably distinguish a deletion from a move it failed to detect.
Of 195 apparent deletions in this history, 83 have a same-named page somewhere else today:
/develop/reference/cluster-specoperate/oss_and_stack/reference/cluster-spec.md/develop/manual/keyspace-notificationsdevelop/pubsub/keyspace-notifications.md/develop/manual/client-side-cachingdevelop/clients/client-side-caching.mdThose fell below git's rename similarity threshold and were recorded as delete-plus-add. And the remaining 112 are no safer — a page can be renamed and relocated at once, which no name-matching heuristic can see.
Publishing that list would tell a consumer to discard a citation that still resolves, which is worse than telling them nothing. So instead the docs now state plainly that a URL missing from the map is not necessarily gone, and that we deliberately don't claim deletions.
A list that's 40% wrong under the first check anyone would run against it isn't worth shipping.
Verification
Full build with a production-style baseURL:
moved_onREF_NOT_FOUNDset already onmain(DOC-6955)data/page-moves.jsonabsent🤖 Generated with Claude Code
Note
Low Risk
Docs build pipeline and published redirect metadata only; optional field when data file is missing, no auth or runtime app changes.
Overview
Adds
moved_onto entries in the publishedredirects.jsonmap when git history can date a real page move, so consumers can tell stale vs recently changed redirects from vanity aliases with no date.A new
build/generate_page_moves.pystep (wired intomake hugo/serve_hugoviapage_moves) scans rename records and writes gitignoreddata/page-moves.json;layouts/index.redirects.jsonmerges those dates into unambiguous redirect entries. CI’s latest build job now checks out withfetch-depth: 0so shallow clones don’t silently ship zero dates; the script warns on shallow clones. AI agent docs clarify that missingmoved_onmeans legacy alias, not undated move, and that deleted pages are intentionally not listed because git can’t reliably separate deletion from undetected moves.Reviewed by Cursor Bugbot for commit dd114ce. Bugbot is set up for automated code reviews on this repo. Configure here.