Skip to content

chore(ci): explain the frozen golden in its own failure message - #589

Merged
ss-o merged 1 commit into
mainfrom
bug-588
Sep 2, 2026
Merged

chore(ci): explain the frozen golden in its own failure message#589
ss-o merged 1 commit into
mainfrom
bug-588

Conversation

@ss-o

@ss-o ss-o commented Sep 2, 2026

Copy link
Copy Markdown
Member

Learning capture from #586, which appended one section to PATTERNS.md and
failed Validate Agent Instructions with nothing but two bare SHA-256 strings.

test_repair_2_consumer_parser_outputs_match_frozen_golden hashes the parsed
output of seven consumer surfaces, PATTERNS.md and .github/README.md among
them. Editing any of them changes the digest, which is the test working
correctly. Nothing recorded that, though: grep -ri golden over AGENTS.md,
runbooks/, decisions/, and .github/instructions/ returns nothing, and the
test file has been amended nine times. #582 hit the same wall on a skill edit.

This attaches the seven paths and the regeneration procedure to the assertion's
msg=, so both appear in the failure output. One argument covers all seven
surfaces, where a per-file comment would cover one each and drift as the list
changes.

Verified by appending a line to PATTERNS.md and re-running the test. The
message renders with every path enumerated, and PATTERNS.md was restored, so
this branch changes one file. The full suite of 99 tests passes, as do
validate-agent-policy.py and the Python linters.

Closes #588

The digest covers seven consumer surfaces, so editing any of them fails
Validate Agent Instructions with nothing but two bare SHA-256 strings. Nothing
in AGENTS.md, runbooks, decisions, or the scoped instructions says which files
feed it or how to regenerate it, so #582 and #586 both discovered the step by
failing CI. Attach the list and the procedure to the assertion, where the
person who needs it is already looking.

Closes #588
@ss-o
ss-o requested a review from a team as a code owner September 2, 2026 12:53
@ss-o
ss-o merged commit cf1fd88 into main Sep 2, 2026
9 checks passed
@ss-o
ss-o deleted the bug-588 branch September 2, 2026 12:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

chore(ci): explain how to regenerate the frozen consumer-parser golden

1 participant