add docs drift check - #7
Open
joe-clickhouse wants to merge 1 commit into
Open
Conversation
| # Claude compares the PR's code changes against the repo's documentation and | ||
| # reports whether a user-visible change is missing a matching docs update. The | ||
| # workflow then applies or removes a `needs-docs` label and upserts a single | ||
| # sticky comment. It is advisory: the check itself always passes unless the |
Collaborator
There was a problem hiding this comment.
Not entirely true, in case of a failure (eg API outage) on the claude step, it'll fail. continue-on-error: true on the Claude step to make it true.
| BODY="${BODY#$'\n'}" | ||
| BODY="${MARKER}"$'\n'"${BODY}" | ||
|
|
||
| HAS_LABEL=$(gh pr view "$PR" --json labels \ |
Collaborator
There was a problem hiding this comment.
I think this will fail as written:
$ gh pr view 7 --json labels --jq --arg l "needs-docs" '[.labels[].name] | index($l) != null'
accepts at most 1 arg(s), received 4
EXIT=1
Instead, something like:
HAS_LABEL=$(gh pr view "$PR" --json labels \
--jq '[.labels[].name] | index(env.LABEL) != null')
Collaborator
|
Maybe make the labeling optional? Empty label = opt out? |
alex-clickhouse
approved these changes
Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add reusable docs drift check workflow
Adds
claude-docs-drift.yml, a reusable workflow that checks whether a pull request's code changes need matching documentation updates. It follows the structure and security posture ofclaude-pr-triage.yml.How it works
A client repo calls this workflow on its pull requests. Claude reads the PR diff and the repo's own rubric file at
.claude/agents/docs-drift-reviewer.md, which defines what counts as user facing surface in that repo and where itsdocs live. Claude is report only and returns schema validated JSON. A deterministic step then applies or removes a
needs-docslabel and keeps a single sticky comment up to date. Every push re-runs the check and reconcileslabel and comment state from scratch, so fixing the docs in a later commit clears the label and marks the comment resolved.
The check is advisory. It always passes unless the caller sets the
fail_on_driftinput, which allows a repo to make it a required check later.Opt in per repo
If the rubric file does not exist on the checked out ref the run is a green no-op. Client repos can adopt the small caller workflow at any time and the check activates only once a maintainer merges a rubric file. clickhouse-connect will be the pilot repo.
Security
Same hardening as the triage workflow. The Claude step has no network tools, no write tools, and no arbitrary Bash. It only runs read only
gh pr view,gh pr diff, andgh issue viewplus file reads on the checkout. All label and comment writes happen in a deterministic step from validated JSON, so prompt injection is limited to a misleading advisory comment. Fork PRs are skipped because they receive no secrets. Maintainers can run them throughworkflow_dispatchafter a sanity look, and on that path the checkout is the default branch so the rubric and docs Claude reads are the trusted versions.Inputs
agent_path.claude/agents/docs-drift-reviewer.mdlabelneeds-docsfail_on_driftfalsemodelmax_turns30pr_numberRequires the
ANTHROPIC_API_KEYsecret and caller permissionscontents: read,pull-requests: write,issues: write.