Skip to content

feat(pr-workflow): add the /mms-evidence orchestrator, runners, and publish gate - #84

Draft
MajorLift wants to merge 1 commit into
mainfrom
jongsun/add/pr-validate-falsifying-test
Draft

feat(pr-workflow): add the /mms-evidence orchestrator, runners, and publish gate#84
MajorLift wants to merge 1 commit into
mainfrom
jongsun/add/pr-validate-falsifying-test

Conversation

@MajorLift

@MajorLift MajorLift commented Jul 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Withdrawn.

@MajorLift MajorLift mentioned this pull request Jul 30, 2026
Closed
@MajorLift MajorLift changed the title feat(pr-workflow): add pr-validate and falsifying-test skills feat(pr-workflow): add pr-validate and falsifying-test skills Jul 30, 2026
@MajorLift
MajorLift marked this pull request as draft July 30, 2026 14:03
@MajorLift
MajorLift marked this pull request as ready for review July 30, 2026 18:15
@MajorLift

MajorLift commented Jul 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Context budget

What this PR costs an agent, measured from an install rather than read from the diff. Three tiers, and only the first is unavoidable.

Skill Frontmatter Selected + refs & knowledge
evidence 1,137 chars ~6,838 tok ~35,592 tok
falsifying-test 814 chars ~1,374 tok ~2,195 tok

Frontmatter is the only tier paid unconditionally — every agent loads it on every run once the skill is installed, used or not, because it is what the agent reads to decide relevance. The 28 skills across the eleven open skill PRs sit at a median of ~1,716 tokens selected and ~1,860 with references followed. All are within the 1,536-character description budget.

Selected is paid only when the agent picks the skill. + refs & knowledge is the ceiling if every bundled reference is then read; it is a worst case, not an expectation.

Method

tools/install --repo metamask-extension --maturity experimental against this branch at eceaf37d8, measured per installed skill directory. Repo overlays are merged into the emitted SKILL.md, so they land in the selected tier rather than being missed by a source-byte count. Token figures are bytes/4 — a proxy for scale, not accounting.

These figures are pinned to the commit above and drift on every push; #96 tracks automating them.

@MajorLift MajorLift changed the title feat(pr-workflow): add pr-validate and falsifying-test skills feat(pr-workflow): add evidence and falsifying-test skills Jul 31, 2026
@MajorLift MajorLift changed the title feat(pr-workflow): add evidence and falsifying-test skills feat(pr-workflow): add /mms-evidence orchestrator and falsifying-test skill Jul 31, 2026
This was referenced Jul 31, 2026
Open
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
@MajorLift
MajorLift marked this pull request as draft August 3, 2026 09:42
MajorLift added a commit that referenced this pull request Aug 3, 2026
@MajorLift
Add five reasoning audits and two diagnostic skills
4fa12d7 
Each encodes a failure that actually shipped, rather than a principle that
sounded right. `unmeasured-join` targets the costliest one — a conclusion
assembled from true facts through a step nobody measured, which survives review
because the facts check out and the join reads as prose. `instrument-check`
requires a positive and a negative control before a measurement counts, after a
mutation runner reported a falsification for a line it never wrote.
`scope-of-search` makes a negative carry the pattern that produced it, after a
grep keyed on a variable name declared absent what its own output printed two
blocks later. `coverage-partition` replaces "the test has power" with which cases
guard which mechanism. `selection-audit` separates a count from the rule that
produced it, after two different selections landed on the same number and the
match closed the question.

`distinguishing-observation` and `observability-gap` point the same discipline at
debugging: design the observation that separates candidates rather than confirms
the favourite, and establish what signal exists on a path before reading more of
it.

The `evidence` links are forward references — that skill ships in #84 and is not
on main yet.
@MajorLift MajorLift mentioned this pull request Aug 3, 2026
Merged
4 tasks
@MajorLift MajorLift changed the title feat(pr-workflow): add /mms-evidence orchestrator and falsifying-test skill feat(pr-workflow): add /mms-evidence orchestrator and red-on-base skill Aug 3, 2026
MajorLift added a commit that referenced this pull request Aug 3, 2026
@MajorLift
Take red-on-base from #84, where it was the odd one out
ca9111f 
Its substance is the same as everything else here: what counts as proof, and how
a proof can look right while testing the wrong thing. Its falsifier — a base
commit that fails for the wrong reason produces an identical exit code and proves
nothing — is the same move `unmeasured-join` and `scope-of-search` make on
different material.

It sat in #84 because it was written alongside the orchestrator, before the
instrument and the reasoning that points it were separate ideas. Every other
engine already ships in its own pull request.

Renamed from `falsifying-test` before the move, because it and `falsify-probe.sh`
shared a root while answering different questions — base-against-branch asks
whether a test is connected to the reported bug, one-commit-with-a-mutated-line
asks whether a suite notices the mechanism going away.
@MajorLift MajorLift changed the title feat(pr-workflow): add /mms-evidence orchestrator and red-on-base skill feat(pr-workflow): add the /mms-evidence orchestrator, runners, and publish gate Aug 3, 2026
This was referenced Aug 3, 2026
Draft
Draft
MajorLift added a commit that referenced this pull request Aug 5, 2026
@MajorLift
Add five reasoning audits and two diagnostic skills
a1f859d 
Each encodes a failure that actually shipped, rather than a principle that
sounded right. `unmeasured-join` targets the costliest one — a conclusion
assembled from true facts through a step nobody measured, which survives review
because the facts check out and the join reads as prose. `instrument-check`
requires a positive and a negative control before a measurement counts, after a
mutation runner reported a falsification for a line it never wrote.
`scope-of-search` makes a negative carry the pattern that produced it, after a
grep keyed on a variable name declared absent what its own output printed two
blocks later. `coverage-partition` replaces "the test has power" with which cases
guard which mechanism. `selection-audit` separates a count from the rule that
produced it, after two different selections landed on the same number and the
match closed the question.

`distinguishing-observation` and `observability-gap` point the same discipline at
debugging: design the observation that separates candidates rather than confirms
the favourite, and establish what signal exists on a path before reading more of
it.

The `evidence` links are forward references — that skill ships in #84 and is not
on main yet.
MajorLift added a commit that referenced this pull request Aug 5, 2026
@MajorLift
Take red-on-base from #84, where it was the odd one out
70ed100 
Its substance is the same as everything else here: what counts as proof, and how
a proof can look right while testing the wrong thing. Its falsifier — a base
commit that fails for the wrong reason produces an identical exit code and proves
nothing — is the same move `unmeasured-join` and `scope-of-search` make on
different material.

It sat in #84 because it was written alongside the orchestrator, before the
instrument and the reasoning that points it were separate ideas. Every other
engine already ships in its own pull request.

Renamed from `falsifying-test` before the move, because it and `falsify-probe.sh`
shared a root while answering different questions — base-against-branch asks
whether a test is connected to the reported bug, one-commit-with-a-mutated-line
asks whether a suite notices the mechanism going away.
@MajorLift
MajorLift force-pushed the jongsun/add/pr-validate-falsifying-test branch from d3d153d to edf65a3 Compare August 6, 2026 10:28
This was referenced Aug 6, 2026
Open
Merged
Open
Open
Open
Open
Open
MajorLift added a commit that referenced this pull request Aug 11, 2026
@MajorLift
Add five reasoning audits and two diagnostic skills
e40328f 
Each encodes a failure that actually shipped, rather than a principle that
sounded right. `unmeasured-join` targets the costliest one — a conclusion
assembled from true facts through a step nobody measured, which survives review
because the facts check out and the join reads as prose. `instrument-check`
requires a positive and a negative control before a measurement counts, after a
mutation runner reported a falsification for a line it never wrote.
`scope-of-search` makes a negative carry the pattern that produced it, after a
grep keyed on a variable name declared absent what its own output printed two
blocks later. `coverage-partition` replaces "the test has power" with which cases
guard which mechanism. `selection-audit` separates a count from the rule that
produced it, after two different selections landed on the same number and the
match closed the question.

`distinguishing-observation` and `observability-gap` point the same discipline at
debugging: design the observation that separates candidates rather than confirms
the favourite, and establish what signal exists on a path before reading more of
it.

The `evidence` links are forward references — that skill ships in #84 and is not
on main yet.
MajorLift added a commit that referenced this pull request Aug 11, 2026
@MajorLift
Take red-on-base from #84, where it was the odd one out
28ecea1 
Its substance is the same as everything else here: what counts as proof, and how
a proof can look right while testing the wrong thing. Its falsifier — a base
commit that fails for the wrong reason produces an identical exit code and proves
nothing — is the same move `unmeasured-join` and `scope-of-search` make on
different material.

It sat in #84 because it was written alongside the orchestrator, before the
instrument and the reasoning that points it were separate ideas. Every other
engine already ships in its own pull request.

Renamed from `falsifying-test` before the move, because it and `falsify-probe.sh`
shared a root while answering different questions — base-against-branch asks
whether a test is connected to the reported bug, one-commit-with-a-mutated-line
asks whether a suite notices the mechanism going away.
This was referenced Aug 12, 2026
Merged
Merged
Merged
Merged
Draft
Merged
MajorLift added a commit that referenced this pull request Aug 20, 2026
@MajorLift
Merge #103: validate cross-skill references in lint-skill-entry
55b6d4d 
Absorbs #103. No file overlap with this branch and no conflict — the two
were adjacent in theme rather than entangled, unlike #94, which shared
test/cli.test.mjs and the same four citations.

Brings three reference checks in the linter: bare lane ids cited without
linking the catalog that defines them, [[snake_case]] wiki links from a
private authoring vault, and `## Related` entries naming a skill that
does not exist on the branch. Plus check-public-refs, which asserts every
repository named in the corpus resolves anonymously, and skill-audit.

Scope note for reviewers: the lane check has no corpus subject today.
Zero matches across 166 skill markdown files here, and zero across 256
non-evidence files on #84's branch, because evidence-catalog.md — the
only place [A-G]N vocabulary exists — lives on #84. It is fixture-tested
and prospective; it fires the day a non-evidence skill cites a lane.

Suite 84/84; lint 52 skills, 0 errors; check-public-refs 6/6 resolve.
Falsifiable: stubbing LANE_ID fails 2 lint tests, restoring passes 33/33.
@MajorLift MajorLift mentioned this pull request Aug 20, 2026
Open
@MajorLift MajorLift mentioned this pull request Aug 31, 2026
Draft
MajorLift added a commit that referenced this pull request Sep 1, 2026
@MajorLift
Route wasted-render work to react-render-delta, the skill that exists
acd099c 
The symptom table and the description both named `react-render-proof`. No skill
of that name exists on `main` or in any open pull request; the render engine is
`react-render-delta`, added by #43 and carried by #84 and #108, and installed
locally as `mms-react-render-delta`.

The other six engines this skill routes to all resolve to skills in open pull
requests, so this was the only wrong name rather than one of three. Checked
against the repository rather than against an install: none of the seven is on
`main` yet, which is a merge-ordering fact and not a defect here.

Nothing validates this today. #103, which would have checked cross-skill
references, was closed as superseded by #87, and #87 is not merged — so this
name would have shipped unflagged.
@MajorLift
MajorLift force-pushed the jongsun/add/pr-validate-falsifying-test branch from e770cf7 to d001682 Compare September 11, 2026 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@MajorLift