fix(planning): word interview questions so yes accepts the recommendation

[kyle-sexton]

Closes #6946

Summary

/planning:interview failed the interview-negative-recommendation-yes-accepts eval case (added in #6986), so this adds the rule the issue's decision held back for that outcome: word each question so a bare "yes" accepts the recommendation.

Fix

• One rule in plugins/planning/skills/interview/SKILL.md, "Recommended answers": when the recommendation is negative, ask "Should X stay out of this change?" rather than "Should we do X?" answered "No".
• Placement: that section sits inside the digest-pinned Stance section. context/gotchas.md is unpinned, but none of the six with-arm traces read it, so a rule there would not reach the model. The maintainer approved recomputing that digest (attended, 2026-10-11): plugins/planning/tests/interview-defenses.test.sh now pins the new Stance section. The rule tightens question wording and does not weaken the no-silent-resolve defense the pin protects.
• Changelog fragment .changes/planning/fix-6946-interview-yes-accepts-rec-d519194b.md (patch).

Verification

claude plugin eval (Claude Code 2.1.296, model claude-opus-5-5, judge haiku, 3 runs per arm, --keep-temp). The yes-accepts-recommendation judge was calibrated first: 18 of 18 runs agreed over 6 labeled samples.

Skill	With-arm score	Judge passes	Without-arm
main (before)	0.33	0 of 3: every run asked "Is the React rewrite part of this task?" with the recommendation "No"	0.33
this branch	0.89	2 of 3; all 3 runs asked "Should the React rewrite stay out of this change?" with the recommendation "yes"	0.44

The one judge FAIL on this branch was on a reply whose wording meets the rule; on reading, it looks like a judge false negative. The validity gate returned VALID for both passes. The noise report says "n too small to call" because the suite has one case.

markdownlint-cli2 reports 0 issues. standards-binding.test.sh passes. interview-defenses.test.sh passes (PASS=172 FAIL=0) with the recomputed Stance digest.

Related

• test(planning): add interview eval case for a negative recommendation #6986 added the eval case.
• test(source-control): run the paid with/without plugin eval for resolve-conflicts #6939 is the companion eval run for resolve-conflicts.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-11T14:35:24.035830Z	7c8f9e3	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[claude]

Claude finished @kyle-sexton's task in 2m 55s —— View job

---

Security review — /review:security-review

• Gather context (PR diff, changed files)
• Invoke /review:security-review per custom instructions
• Post findings

No security issues found.

Scope reviewed (only files changed in this PR, diffed against origin/main):

• .changes/planning/fix-6946-interview-yes-accepts-rec-d519194b.md — new changelog fragment, prose only.
• plugins/planning/skills/interview/SKILL.md#L174-L178 — adds one rule ("word each question so yes accepts the recommendation") inside the Recommended answers section. This is a pure addition, not a deletion/narrowing/softening of an existing instruction, so the instruction-surface lens (docs/conventions/instruction-exception-register/README.md) doesn't flag it — there's nothing removed to check against the six Gate 0 classes.
• plugins/planning/tests/interview-defenses.test.sh#L498 — updates the pin_section SHA-256 digest for the Stance section to match the new line added in SKILL.md. The test's own header comment documents this as the expected, attended procedure for a load-bearing pin: "re-read the defense, confirm it still holds, then update the skill body and this suite in one change" — which is exactly what the companion commit test(planning): update the Stance digest for the yes-accepts-recommendation rule does, alongside the rule addition it corresponds to. No content inside the digested region changed other than the new, additive line; no defense text was weakened. This isn't the "unattended run silently updates a pin" case the test guards against.

No injection surfaces, credential/secret handling, auth/access-control logic, or GitHub Actions workflow changes are present in this diff — it's confined to a skill's markdown instructions, a test's pinned hash, and a changelog fragment.

[github-actions]

claude-security-review has reviewed this pull request through 10cefc2; a later push is reviewed from there.

[claude]

Claude finished @kyle-sexton's task in 4m 51s —— View job

---

/review:code-review

• Gather context (diff vs origin/main, changed files, surrounding SKILL.md/test conventions)
• Understand the request (run the CI code-review lane per custom instructions)
• Review plugins/planning/skills/interview/SKILL.md rule addition
• Review plugins/planning/tests/interview-defenses.test.sh digest update
• Review changelog fragment
• Post findings

Summary

Scope: the 3 changed files (.changes/planning/fix-6946-interview-yes-accepts-rec-d519194b.md, plugins/planning/skills/interview/SKILL.md, plugins/planning/tests/interview-defenses.test.sh), diffed against origin/main.

1 finding, posted inline on interview-defenses.test.sh#L498: the test file's own header mandates recording, next to each pin_section call, why an addition inside that digested region doesn't weaken the defense it protects — every prior addition to this exact Stance-section digest has such a comment (lines 471-493). The new rule added at SKILL.md#L177 has none — the reasoning currently lives only in the PR description, not in the file future readers will consult.

No other correctness issues found:

• The new rule's wording ("Should X stay out of this change?" vs. "Should we do X?" answered "No") is scoped correctly to the polar/yes-no case the eval failure actually exercises; it doesn't conflict with the multi-option Alternatives pattern or the accept-shorthand mechanics elsewhere in the Stance section.
• The section the digest covers (## Stance... through ## The interview loop) already spanned the Acceptance-criteria-capture subsection before this PR, per the test's own boundary markers — the hash change is consistent with that pre-existing scope, not a new leak.
• Changelog fragment is prose-only, matches the expected bump/### Changed shape.

Not independently verified: I could not execute interview-defenses.test.sh or recompute the SHA-256 digest myself — this sandbox's permission mode blocked script execution and redirection needed to do so. The PR's own verification claims (test suite PASS=172 FAIL=0, markdownlint-cli2 clean, eval scores) are author-claimed and unverified by me; ci-status and the test suite's own run will be the actual gate on whether the recomputed digest is correct.

[claude]

Claude finished @kyle-sexton's task in 1m 8s —— View job

---

Security review — /review:security-review

• Gather context (PR diff, changed files)
• Invoke /review:security-review per custom instructions
• Post findings

No security issues found.

Skip-gate check: HEAD 7c8f9e3 is a merge of origin/main into this branch. Diffed against the
previously-reviewed commit (4bc4dd9), the three changed files are byte-identical — the merge
pulled in unrelated upstream commits only. The prior review's conclusion still applies; I
independently re-read the diff against origin/main to confirm before relying on that.

Scope reviewed (files changed in this PR, diffed against origin/main):

• .changes/planning/fix-6946-interview-yes-accepts-rec-d519194b.md — new changelog fragment, prose only, no executable content.
• plugins/planning/skills/interview/SKILL.md#L177 — adds one rule ("word each question so yes accepts the recommendation") inside the Recommended answers section, a pure addition with no removed - lines in the hunk. Under the instruction-surface lens (docs/conventions/instruction-exception-register/README.md, present in this checkout), only a deletion/narrowing/softening of a standing instruction requires checking against the six Gate 0 protected classes; an added rule has nothing to check against, so this isn't a candidate.
• plugins/planning/tests/interview-defenses.test.sh#L498 — updates the pin_section SHA-256 digest for the Stance section to match the new line added in SKILL.md. This is the documented attended procedure for a load-bearing pin (re-read the defense, confirm it still holds, update skill body and suite together), matched here by the companion commit test(planning): update the Stance digest for the yes-accepts-recommendation rule. No content inside the digested region was removed or weakened, only the one additive line — not the "unattended run silently updates a pin" case this test guards against.

No injection surfaces, credential/secret handling, auth/access-control logic, or GitHub Actions workflow changes are present in this diff — it remains confined to a skill's markdown instructions, a test's pinned hash, and a changelog fragment.

[github-actions]

claude-review has reviewed this pull request through 10cefc2; a later push is reviewed from there.

[claude]

Claude finished @kyle-sexton's task in 1m 4s —— View job

---

/review:code-review

• Gather context (incremental diff since 7c8f9e3, scoped to plugins/planning/tests/interview-defenses.test.sh)
• Invoke /review:code-review per custom instructions
• Post findings

Summary

Scope (per lane instructions): only what changed in plugins/planning/tests/interview-defenses.test.sh since the last review at 7c8f9e3, i.e. commit 10cefc2. Every other file was reviewed at 7c8f9e3 and is unchanged since.

No findings. The diff adds exactly the explanatory comment the prior review on this lane requested — interview-defenses.test.sh#L494-L496:

# The yes-accepts-recommendation rule words a negative recommendation as "Should X stay out of
# this change?" so "Q<N> yes" and "accept all" mean the recommended answer. It changes how a
# question is phrased, not what resolves it: an unanswered question still resolves nothing.

Checked this against the rest of the review:

• Matches the file's own convention: every other addition inside this digested Stance section (lines 471-493) carries exactly this kind of one-paragraph comment explaining why it doesn't weaken the no-silent-resolve defense. This new entry follows the same shape and sits directly above the pin_section call it documents.
• Accurate to the rule it describes (SKILL.md#L177, unchanged since 7c8f9e3): "Word each question so yes accepts the recommendation... ask 'Should X stay out of this change?', never 'Should we do X?' answered 'No'."
• "Q yes" / "accept all" are established shorthand terms elsewhere in this skill's docs (context/loop.md, context/gotchas.md), not invented here, so the terminology is consistent.
• No digest value changed in this diff (that update already landed in 4bc4dd9, reviewed previously) — this commit is comment-only, as claimed.

This closes out the one open review comment from the prior pass; no new issues.

[claude]

Claude finished @kyle-sexton's task in 35s —— View job

---

/review:security-review

• Gather context (incremental diff since 7c8f9e3, scoped to plugins/planning/tests/interview-defenses.test.sh)
• Invoke /review:security-review per custom instructions
• Post findings

No security issues found.

Skip gate: the incremental diff since 7c8f9e3 (.claude-lane/incremental.diff) touches only
plugins/planning/tests/interview-defenses.test.sh,
and the change is three new comment lines only — no code, no digest value, no assertion changed:

+# The yes-accepts-recommendation rule words a negative recommendation as "Should X stay out of
+# this change?" so "Q<N> yes" and "accept all" mean the recommended answer. It changes how a
+# question is phrased, not what resolves it: an unanswered question still resolves nothing.

This is a pure addition (no - lines), so the instruction-surface lens (deletion/narrowing/softening
of a standing instruction) doesn't apply — there's nothing removed to check against the register's
protected classes. The content is exactly the self-documentation this suite's own convention asks
for at this digested pin — the inline finding from the prior review round
(comment, test file line 501)
asked for a comment here explaining why the new Stance-section rule doesn't weaken the
no-silent-resolve defense, and that's what this hunk adds. The pin_section hash itself is
unchanged in this increment (it was already updated and reviewed at 7c8f9e3).

No injection surfaces, credential/secret handling, auth/access-control logic, or GitHub Actions
workflow changes are present in this increment — it is a comment-only addition to a shell test
file.