Skip to content

add docs drift check - #7

Open
joe-clickhouse wants to merge 1 commit into
mainfrom
joe/claude-docs-drift
Open

add docs drift check#7
joe-clickhouse wants to merge 1 commit into
mainfrom
joe/claude-docs-drift

Conversation

@joe-clickhouse

Copy link
Copy Markdown

Add reusable docs drift check workflow

Adds claude-docs-drift.yml, a reusable workflow that checks whether a pull request's code changes need matching documentation updates. It follows the structure and security posture of claude-pr-triage.yml.

How it works

A client repo calls this workflow on its pull requests. Claude reads the PR diff and the repo's own rubric file at .claude/agents/docs-drift-reviewer.md, which defines what counts as user facing surface in that repo and where its
docs live. Claude is report only and returns schema validated JSON. A deterministic step then applies or removes a needs-docs label and keeps a single sticky comment up to date. Every push re-runs the check and reconciles
label and comment state from scratch, so fixing the docs in a later commit clears the label and marks the comment resolved.

The check is advisory. It always passes unless the caller sets the fail_on_drift input, which allows a repo to make it a required check later.

Opt in per repo

If the rubric file does not exist on the checked out ref the run is a green no-op. Client repos can adopt the small caller workflow at any time and the check activates only once a maintainer merges a rubric file. clickhouse-connect will be the pilot repo.

Security

Same hardening as the triage workflow. The Claude step has no network tools, no write tools, and no arbitrary Bash. It only runs read only gh pr view, gh pr diff, and gh issue view plus file reads on the checkout. All label and comment writes happen in a deterministic step from validated JSON, so prompt injection is limited to a misleading advisory comment. Fork PRs are skipped because they receive no secrets. Maintainers can run them through workflow_dispatch after a sanity look, and on that path the checkout is the default branch so the rubric and docs Claude reads are the trusted versions.

Inputs

Input Default Purpose
agent_path .claude/agents/docs-drift-reviewer.md Repo relative path of the rubric
label needs-docs Label applied while drift is unresolved
fail_on_drift false Fail the job when drift is found
model action default Optional model override
max_turns 30 Agent turn budget
pr_number empty Required for workflow_dispatch callers

Requires the ANTHROPIC_API_KEY secret and caller permissions contents: read, pull-requests: write, issues: write.

# Claude compares the PR's code changes against the repo's documentation and
# reports whether a user-visible change is missing a matching docs update. The
# workflow then applies or removes a `needs-docs` label and upserts a single
# sticky comment. It is advisory: the check itself always passes unless the

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not entirely true, in case of a failure (eg API outage) on the claude step, it'll fail. continue-on-error: true on the Claude step to make it true.

BODY="${BODY#$'\n'}"
BODY="${MARKER}"$'\n'"${BODY}"

HAS_LABEL=$(gh pr view "$PR" --json labels \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will fail as written:

$ gh pr view 7 --json labels --jq --arg l "needs-docs" '[.labels[].name] | index($l) != null'
accepts at most 1 arg(s), received 4
EXIT=1

Instead, something like:

HAS_LABEL=$(gh pr view "$PR" --json labels \
  --jq '[.labels[].name] | index(env.LABEL) != null')

@alex-clickhouse

alex-clickhouse commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Maybe make the labeling optional? Empty label = opt out?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants