Skip to content

IC3 employer gate block: thresholds, partitions, and rulings (draft for referee) - #230

Draft
vahid-ahmadi wants to merge 94 commits into
masterfrom
c3-gate-block-draft
Draft

IC3 employer gate block: thresholds, partitions, and rulings (draft for referee)#230
vahid-ahmadi wants to merge 94 commits into
masterfrom
c3-gate-block-draft

Conversation

@vahid-ahmadi

@vahid-ahmadi vahid-ahmadi commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

DRAFT FOR REFEREE — NOT RATIFIED.

This PR is the referee control document for IC3:
docs/design/ic3_employer_gate_block.md.

It does not edit gates.yaml, lock IC3, authorize an audit draw,
or authorize a candidate run.

Exact prerequisite composition

This branch composes these current prerequisite heads so the control
document and its cited interfaces can be reviewed together:

#212 and #223 are composed at those exact heads, but composition is
not readiness, approval, merge, or ratification. #212's three
builders were rerun from the exact official Census-hosted vintages:
every previously recorded measured value reproduced exactly, current
source-input and measurement-environment sidecars are pinned, and E9
now records distinct-person counts (stay 16,286; J2J 524). The resolved
strict-staging/raw-provenance/count blockers are no longer asserted.
#212 still requires current-head approval and merge, and its
deployment-scale conversion and threshold choices remain
REFEREE/PENDING. #223 remains a pre-lock anchor pending current-head
approval and merge. Temporal-basis, deployment conversion, and threshold
choices remain REFEREE/PENDING.

What this revision reconciles

  • uses final IC terminology and design filenames;
  • distinguishes E4 pair units from E5 maximal-run units and keeps
    their populations, propensities, and weights separate;
  • records both Cross-wave job-ID check: disclosed re-analysis, verdict by population (pre-lock, #230 §6) #235 seam populations without choosing the operative
    population or correction bands;
  • fixes the scale direction: inflate person-model cells to job-based
    QWI/J2J references; never deflate the administrative references;
  • limits E11 to aggregate-flow evidence and explicitly forbids using
    an E11 pass as proof of a worker–firm link;
  • records the E12 two-tier boundary: a future disjoint aggregate
    margin audit may certify only grouped observables, while strong
    linkage, sorting, variance, firm-effect, coworker, and spillover
    claims remain deferred/no-go;
  • records only decisions already made by Vahid in GitHub comments;
    every unmade choice is enumerated as REFEREE/PENDING in §13;
  • records exact current artifact, builder, input-sidecar, and
    measurement-environment pins while separating branch composition
    from approval and merge provenance.

Verification

The remaining decisions include #235 population/bands, #212
scale/deployment conversion and thresholds, #236 audit numerics and
draw/blinding mechanics, #223 temporal-basis and thresholds, and the
later mechanical locked: false draft block. Review is requested on
the current head; the PR remains draft.

cc @daphnehanse11 @MaxGhenis

daphnehanse11 and others added 10 commits July 15, 2026 13:32
Person-disjoint half-vs-half floors on the pu2023 job-month panel,
the disability-floor convention (sha256 half-splits, seeds 0-4,
per-cell |log rate ratio| mean/sd, WPFINWGT-weighted). E4: monthly
same-employer retention by age band x sex (99.0% at 16-24 rising to
99.7% at 35-44; floors 0.001-0.002). E5: maximal same-employer run
lengths (full-year share 46.7% at 16-24 to 86.7% at 45-64; floors
0.01-0.05). No thin cells. Explicitly DRAFT: no thresholds, C3 not
locked, seam caveat recorded — the seam-vs-J2J reconciliation is a
separate required artifact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Completes the tenure side of the Workstream A floor battery: person-
disjoint half-vs-half floors for the E3 moment (tenure P25/P50/P75
by BLS age band) on all three staged January supplements.

Method finding worth the referee round's attention: reported tenure
heaps hard on integers, so absolute quantile gaps between halves are
EXACTLY zero in 36/63 cells (both halves' quantiles land on the same
heap) — a degenerate basis for a quantile-error-vs-floor criterion.
Each cell therefore also carries a weighted-ECDF max-gap floor
(1.5-5.6% in 2024, tightest in the thick prime-age cells), which is
smooth under heaping; C3 can choose between the formulations with
both on the record.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
E8 (nonemployment spells, full-year persons, censoring-free draft
restriction recorded): any-nonemployment share 41.5% (16-24) falling
to 8.5% (45-54) with the retiree uptick at 65+; floors |log ratio|
0.05-0.14; no thin cells. E9 (earnings change by transition):
monthly person-transition mix stay 97.7% / entry 1.01% / exit 0.94%
/ j2j 0.35%, consistent with the seam-reconciliation rates; j2j
median log-change +0.226 (IQR 1.07, n=545, not thin).

Second heaping finding for the referee round: within-job SIPP
monthly earnings are mostly wave-constant, so the STAY median
log-change heaps at exactly 0 and its floor is degenerate — same
failure class as the tenure quantile heaping. E9-stay thresholds
should be stated on the IQR (0.0656, floored at 0.004) or a
distributional distance; caveat recorded in the artifact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…evision in floor metadata

- Merge origin/master (picks up the current sipp_jobs.py reader; no
  conflicts, PR-side additions preserved)
- Record thin-flag units per artifact (E4 counts distinct persons,
  E5/E8 rows==persons, E9 transition pairs, E3 CPS records) in the
  builder docstrings, build() metadata, and hand-edited artifact
  metadata (staged microdata unavailable; measured floors untouched)
- Record the reader commit in effect (sipp_jobs.py 75d30dd for the
  SIPP floors, cps_tenure.py 5c9e5e6 for the tenure floors)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Codex gpt-5.6-sol <noreply@openai.com>
Co-Authored-By: Codex gpt-5.6-sol <noreply@openai.com>
…p_jobs guard, tier_counts)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ons, rulings)

Joint pre-registration draft for the employer gate block, synthesizing
issue #192, ADR 0003/0004 (as amended by the PR #224 review), and the
draft floor/seam artifacts on PRs #212/#214/#223/#228. DRAFT FOR
REFEREE - NOT RATIFIED; edits no gates.yaml cell.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 17, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
social-security-model Ready Ready Preview Jul 31, 2026 9:09am

Request Review

…§6)

The blocking check for the seam ruling (ADR 0004 referee item 7;
#214 concept-delta 5). Verdict rule pre-registered in the script
before the numbers were seen. Results: gross ID survival across the
pu2022->pu2023 boundary 90.55%; re-key signature (same industry +
class of worker + earnings within 20%) among seam separators-to-
employment 17.5% vs a 2.4% within-wave coincidence baseline; scaled
to all seam separations (38.1% are exits to nonemployment, which
cannot be ID artifacts), the implied ID-artifact share of the 9.45%
seam rate is <= 9.4% — under the 15% PASS bar. At least ~90% of the
seam contrast is real seam-bunched separation; the #214 ruling's
conditional check is satisfied.

Implementation note, disclosed: the first run inner-joined the
next-month jobs frame and silently dropped exits to nonemployment
(printing a 6.06% conditioned seam rate and a
PASS_WITH_CORRECTION_BAND verdict against the wrong denominator);
the fix restores the documented design via the person-month
universe and reproduces #214's 9.45%/1.77% exactly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@daphnehanse11

Copy link
Copy Markdown
Collaborator

Workstream A referee response (§13 items in my lane, plus the §6 pre-lock artifact — delivered)

Read the full draft against the committed artifacts. Position: the block is ready to lock once the pre-lock artifacts land, with the answers below. The §6 blocking check is no longer open — it is built, run, and PR'd (see item 8).

Q1 — threshold policy: adopt as proposed (max(floor mean + 4·sd, substantive tolerance), uniform k=4). Per-gate k invites exactly the threshold-shopping the ceremony exists to prevent; the gate-1 precedent band brackets 4 at its conservative end, and the aggregated-sd argument holds. The five A-side substantive tolerances (E3 0.02 ECDF; E4 0.005; E5 0.05; E8 0.10; E9-stay 0.02 IQR-gap) are adopted as proposed — noting, deliberately and on the record, that for E4's youngest cells the floor+4sd term exceeds the 0.005 tolerance and correctly dominates, and that the E4 tolerance being the operative bound elsewhere is the intended anti-understatement posture, not an accident.

Q4 — E3: adopt the ECDF max-gap. It is my degeneracy finding and the quantile alternative would need an arbitrary patch over 36 exact zeros. One refinement for the amendment PR: retain the quantile gaps as report-only diagnostics (they are the interpretable-in-years view consumers will actually read), gating solely on the ECDF stat.

Q5 — E9-stay: adopt IQR-only. A distributional distance cannot gate at first lock without violating floors-before-thresholds; if promoted later it takes a new floor build plus its ADR 0004 audit at promotion time, per the §9 no-inheritance rule.

Q6 — imputed-band adjudicability (blocking): no admissible truth frame exists, and the right resolution is permanent report-only status with a recorded degradation rule, not deferral. LIFE-M-style precision auditing adjudicates links to records that exist; a QRF-imputed firm-size band on a CPS host is a model output whose target (administrative enterprise size, per frozen C2 semantics) was never measured for that person — not by NOEMP (a noisy self-report), not by SIPP (establishment, a different object), not by any representative joint source (ADR 0003's own finding). A hand coder cannot adjudicate a truth that has no record. Proposed degradation rule for C3 to record verbatim: "Cells conditioning on imputed firm-size bands are validated distributionally (calibration fit to SUSB margins plus held-out-axis stability) and are report-only in every phase; they gate only if an external person-level truth source materializes, at which point they enter through the standard promotion ceremony (new floor + ADR 0004 audit)." This also closes the same question for E12 consistently with its no-go rule. First lock scoping them out is correct; make the permanence explicit rather than implicit.

Q8 — the seam ruling: ratify, with the blocking check now delivered. The cross-wave job-ID consistency check is committed as #235 (runs/crosswave_jobid_check_draft_v0.json), with its verdict rule pre-registered in the build script before the numbers were seen. Verdict: PASS — gross ID survival across the pu2022→pu2023 boundary is 90.55%; the re-key signature (same industry + class of worker + earnings ±20%) among seam separators-to-employment runs 17.5% against a 2.4% within-wave coincidence baseline; and since 38.1% of seam separations are exits to nonemployment (structurally incapable of being ID artifacts), the implied artifact share of the 9.45% seam rate is ≤9.4%, under the 15% PASS bar. At least ~90% of the seam contrast is real. The artifact also reproduces #214's 1.77%/9.45% rates exactly from independent code paths. The §6 ruling (J2J levels / SIPP persistence / seam-aware estimation) is therefore ratifiable on its stated condition; recommend recording the ≤9.4% bound as a footnote on the seam figures rather than a correction band.

Q11 — jobs→persons factor: adopt, with the source pinned now. Propose: BLS CPS multiple-jobholders-as-percent-of-employed (series LNS12026620), reference-year annual average, vintage recorded in the amendment PR. Direction note for the record: the factor inflates person-denominated model cells to job-denominated QWI/J2J comparisons, never the reverse.

Q14 — thin-cell rules: confirm 200/half on the SIPP side (it is my draft constant; the rationale worth recording: at p≈0.5 a 200-person half gives a half-vs-half |log ratio| sd near 0.14 — floors that wide gate nothing, so 200 is where flooring stops being meaningful). The 10,000-job B-side minimum is Workstream B's to justify; no objection.

Q15 — ASEC reference-period mismatch: confirm carriage as the label-misalignment note on affected cells, per the frozen ADR 0003 conditioning-DAG text.

Ownership acceptances (§9.3): Workstream A accepts the E4/E5 SIPP-internal adjudication frame and coder-panel operation, and will deliver the minimal-audit manifest as its §12.2 pre-lock artifact. The E4/E5/E8 formulations, the E10 never-weakens hard rule, and the §10 partition lists are endorsed as drafted.

On the remaining items (Q2/Q3/Q7 numerics/Q9/Q10/Q12/Q13): no A-side objection to the recommendations as drafted; Q2's ex-pandemic basis and Q9's margins-gated E11 both follow the same floors-before-thresholds logic applied consistently, and Q12's per-cell caveat avoids a re-floor for a second-order scope delta the #228 extracts already quantify.

🤖 Generated with Claude Code

@daphnehanse11

Copy link
Copy Markdown
Collaborator

Second Workstream A pre-lock artifact delivered: the E4/E5 minimal-audit manifest is up as #236 — the full ADR 0004 design registered at first-lock scope (frame = the #235 population, blinded ID-masked coding, conservative indeterminate handling registered pre-labels, draw seed pinned, leakage freezes), with §6.1 numeric slots left as REFEREE items and a worked binomial example for calibration of that discussion. With #235 and #236, everything §12.2 assigns to Workstream A pre-lock is on the table; remaining are the E2 sex×age floor build and the v1 floor promotions. 🤖 Generated with Claude Code

@MaxGhenis

Copy link
Copy Markdown
Contributor

C3 gate-block referee round — adversarial review (round 1)

Filed by the campaign gate-ceremony referee, per the referee round this draft requests. Not a workstream position. Standard applied: the gate_m6 ceremony (gates.yaml gate_m6 block; docs/amendments/gate_m6_amendment_1_closed_domain_floors.md) and the locked pre-registration rules this repo's every gate has passed through.

As-reviewed pins (all citations below are at these SHAs; branch refs are mutable, so this list is the round's evidence record): #230 @ 1f87c6a (docs/design/c3_employer_gate_block.md, sole file — gates.yaml untouched, confirmed), #212 @ 5346b3b, #214 @ ffb992a, #223 @ f946311, #228 @ a8bb7be, #235 @ 87788eb, #236 @ 3b1af76, #224/ADR 0004 @ ee58b30, master @ 004c57d.

Verdict: NOT RATIFIABLE AS DRAFTED — 5 BLOCKING, 6 SHOULD-FIX

The evidence base underneath is substantially real: every floor number, degeneracy count, and artifact claim I recomputed reproduced exactly (register in Notes). The blocks are structural — partition integrity, ceremony mechanics, contract provenance, and hand-set operative numbers — not arithmetic.


BLOCKING

B1 — E6's gated cells sit inside the calibration span; §10's "explicit lists" omit E6 and E7 entirely; E1 (§13 item 3) fails the same test

  • docs/design/c3_employer_gate_block.md:452-454 registers as calibration cells: "QWI flow margins: firm-size × sector hire and separation rates (qwi_us_firmsize_sector_2015on.csv)".
  • :91 and :197-207 declare E6 gated on the firm-size margin of hire/separation rates from the same file and variable. The size margin is a job-weighted aggregate of the calibrated size×sector cells — a linear functional of calibration targets. Under calibration convergence E6-margin passes by construction; under non-convergence it measures calibration residual. Either way it certifies nothing about the model.
  • The draft's own reasoning is inverted at :205-206: the sector×size cells — which are literally the calibration targets — are demoted to report-only for being "close to the calibration margin," while their deterministic aggregate (where miss terms average out) stays gated.
  • §10 (:442-474) — presented as "the explicit lists that lock with C3" — never mentions E6 or E7 on either side. An enumeration that omits two gates §3 declares gated is a partition-integrity failure on its face: no mutation test over these lists could catch an E6 cell change, because E6 has no cells registered.
  • The frozen contract forbids this: ADR 0003 :157-161 — "Cells used in microcalibrate must be pre-registered as disjoint from gate cells," and its firm-side gate enumeration is "E1/E2/E7/E11" — E6 is not in the frozen held-out list at all.
  • E1, same structure (the draft asks at :465-472 / §13 item 3; here is the answer): the "coarsened BDS-partition margin, which calibration does not target" is targeted up to an exact linear map. Per Pre-IC3 noise floors: the Workstream B aggregate-side battery, promoted to v1 (E1, E2, E6/E7, E11; E11-detail/E12 gaps recorded) #223's committed groups, 1_9 = LT10 (exact) and 10_19+20_99 = B10_49+B50_99 (exact once 20_99 is kept whole), so the coarsened national margin is a merge of the canonical SUSB margins calibration consumes. Scoring that statistic against a BDS/SUSB reference tests source agreement and calibration convergence, neither of which a candidate can influence.
  • House precedent decides the disposition: gate_m6 classifies exactly this trivially-passable class as report-only — gates.yaml not_certified.stock_margins ("the trivially-passable-stock problem").

Required: (i) E1-margin and E6-margin move to report-only/margin class at first lock (with the E1 CV/BDS composite and E6 floors retained as published diagnostics), OR calibration stops consuming QWI flow margins (a joint-PR change to the frozen ADR 0003 partition), OR a genuinely held-out E6/E1 statistic with a committed floor is registered. (ii) §10 re-issued as an exhaustive E1–E12 partition: every gated cell family listed with its disjointness rationale, every calibration input listed, and an ONLY quantifier binding all fitting stages — §10 currently constrains microcalibrate alone, while #192 phase 1 has hazards "calibrated to QWI/J2J"; hazard-calibration consumption is unregistered, which leaves the E2 sex×age and E11-margin holdouts unenforceable as drafted. (iii) The draft YAML (B2) carries the lists in machine-checkable form so a dropped/added/renamed cell fails a test.

B2 — The ceremony is missing the refereed verbatim block and the post-fix verification step

:501-521 (§12): prose draft → referee round → amendment PR carrying "referee-resolved numbers," cross-approved by the two workstreams → merge = ratification. Two deltas from the house standard:

  1. No refereed block text. gate_m6 locked by carrying a committed draft block YAML "VERBATIM with exactly the lock-time deltas" (gates.yaml:5327-5332; docs/design/gate_m6_block_draft.yaml). Here the referee is asked to ratify prose plus PROPOSED numbers, and the actual gates.yaml block text appears for the first time in the amendment PR — transcription errors, silent cell additions, and derivation drift would never have been refereed. E2's cells and floors don't even exist yet (:135-139), so the round cannot currently review the majority of what will lock.
  2. No verification step. The campaign ceremony is draft → adversarial referee → fixes → verification → ratify-by-merge; M6 ran a referee re-check before its flip ("re-check PROCEED TO LOCK," comment 4959310892, cited in the gate_m6 ceremony narrative) and its amendment pinned builder commits, artifact SHA-256s, and the complete authorized edit surface (docs/amendments/gate_m6_amendment_1_closed_domain_floors.md:22-36). §12.3's workstream cross-approval is a party check, not the referee verification.

Required: add to §12: a committed docs/design/c3_employer_gate_block_draft.yaml (enumerated cells, derivations blocks in the tests/test_gates_derivations.py pattern, locked: false) as a pre-lock artifact refereed in this round's continuation and carried verbatim at the flip; an explicit referee verification comment on the amendment PR before merge; and an M6-amendment-style lock table in the amendment PR (builder script commits, artifact SHA-256s, edit surface, guard tests including cell-count mutation pins).

B3 — The block's controlling contract and evidence base are mutable references; §12.2 omits the merges

A locked gate block whose normative ADR is an unmerged branch file amended by comment thread, over evidence on force-pushable branches, is not a locked contract. Required: fold the three adopted changes into ADR 0004's text on #224; extend §12.2 with "#224 and every cited evidence PR merged to master; every consumed artifact SHA-256-pinned at its merge commit." (The pin list at the top of this comment freezes what this round reviewed.)

B4 — #230 silently drops ADR 0004 §3 (linkage-bias reweighting) — the third pillar of #224 — from the block and from §13's "complete" enumeration

#224's requirements are three-part by its own title: precision floors, adjudication samples, linkage-bias reweighting. §9 (:404-440) operationalizes the first two and never mentions the third — no propensity registration, no weighted/unweighted publication rule, no operative-version pre-registration for any link-consuming cell. §13 (:523-566), which claims to enumerate "every decision this draft leaves to the referee," carries ADR 0004 §6.1 (item 7, :545-547) but omits §6.4 entirely (ADR 0004 :364-367: target reference population, propensity model and observables, overlap/balance tolerances, trimming rule, and "the pre-registered operative weighting for E4, E5, E9, E11, and E12"). ADR 0004 §3.4 (:247-258) requires every link-consuming E-cell to publish weighted and unweighted with the operative version chosen before candidate results are seen, and §1.2 (:90-96) requires the full floor registration (assignment, universe, link unit, P_floor, confidence, strata, pooling, failure disposition) before thresholds are named for consuming cells. First-lock scoping to E4/E5 narrows the strata; it does not repeal §3 — E4/E5 are link-consuming gated cells.

Required: §9 gains a §3-instantiation for the first-lock scope (E4/E5 analysis units, candidate observables X, the applicable weight construction, and the operative-version slot), and §13 adds the §6.4 registrations as referee items. Nothing here needs new data; it needs the omitted half of the adopted contract restored before anything locks against it.

B5 — Hand-set substantive tolerances are the operative thresholds across much of the block, mostly without rationale; §4's register misstates the E9-stay degeneracy

Under threshold = max(floor mean + 4·sd, substantive_tolerance) (:58-77), the hand-set term is the binding constraint wherever floors are tiny or degenerate — most E4 cells (floors 0.0003–0.0020 vs tolerance 0.005, verified in #212), and E9-stay entirely: the stay IQR floor is itself degenerate — sipp_e8_e9_floors_draft_v0.json e9_transitions.earnings_change.stay.floor_abs_iqr_gap = {mean: 0.0, sd: 0.0} — so the recommended "IQR-only" stay gate is 100% the hand-set 0.02. Yet §4 (:282-295), whose stated purpose is listing "the committed degeneracies the block must not gate on directly," lists only the stay median (:288-289) and presents the IQR as the non-degenerate escape. It is not; it is a second degenerate floor, disclosed inline at :250 but with no stated basis for 0.02.

Rationales on the record: E1 5% (ECPS convention, :123-125), E3 0.02 (largest adult-band floors, :165-166), E4 0.005 (anti-understatement posture, :175-179), and — via the Workstream A response — the SIPP thin rule. No stated basis: E2 10% (:150), E5 0.05 (:193-194), E6 10% (:207), E7 5% (:224), E8 0.10 (:234-235), E9-stay 0.02 (:250-251), and the B-side 10,000-job thin flag (#223: "draft choice"; §13 item 14). The campaign rule is exact: a hand-set number may gate only as an explicitly disclosed policy choice with rationale.

Required: a per-tolerance basis register in the draft (what substantive error each bound represents and why that magnitude), §4 corrected to record the stay-IQR floor degeneracy, and for E9-stay either a defended 0.02 or demotion of the stay cell to report-only alongside its median. A justification for the 10,000-job thin flag closes §13 item 14's B side.


SHOULD-FIX

S1 — Gate-1 k-precedent is misquoted. :63-67 claims "locked gate-1 rules use k in 4.2–8." The locked derivations use k = 4.2 (gates.yaml:221), 8 (:297), 1.8 (:327), 1.9 (:333), 4 (:493), plus negative/band ks for minima and two-sided metrics (:328,330,332,347). The band is 1.8–8, with two locked ks below the proposed 4. The k=4 choice survives on the aggregated-sd argument alone — and note the SIPP-side floors aggregate 5 seeds exactly as gate-1's did, so the "aggregated vs seed-level" distinction only carries for the 24–36-pair B-side. Restate honestly; the Workstream A response of 13:17 repeats the misquote and should be corrected by the same edit.

S2 — Scoring frame and floor scale are unregistered. E4/E5/E8/E9 reference "SIPP holdout" (:89-94); §5 sketches "person-disjoint holdout" (:301-306) with no split machine, fraction, or seed; nothing asserts the gate-time scoring population matches the half-sample scale the floors were measured at; nothing requires phase-1 training to consume only the holdout's complement, with a leakage guard test. Gate-1's own round-1 amendment was precisely a wrong-scale floor caught pre-lock (gates.yaml:127-133: floor at ~51k windows/side vs ~20k-window scoring). Register split machine/fraction/seed and a per-family floor-scale = scoring-scale assertion in the draft YAML; treat as lock-blocking if still absent at verification.

S3 — #235's verdict-rule provenance is attested, not demonstrated; the bands are hand-set; pin the excess definition. The single commit 87788eb carries rule and result together, and the PR body discloses a prior run yielding "a different verdict against the wrong denominator" — so "pre-registered before the numbers were computed" is not commit-order-verifiable, and the 15%/30% bands have no stated basis. What could be verified, was: every derived field in runs/crosswave_jobid_check_draft_v0.json recomputes exactly from its committed counts (seam 1023/10828 = 0.0945; implied artifact share max(0, 111/633 − 85/3524) × 633/1023 = 0.0936); the verdict follows the committed rule with margin; the result is insensitive to the baseline construction (the unconditional variant gives 0.0960 — also PASS); and the within-wave frame reproduces #214's population exactly (jobs 384,747 = the sum of #214's month-pair tables; seps 6,800; seam 10,828/1,023 — recomputed independently). So the PASS stands on its numbers; the ceremony deficit is real. Required at v1: input pu-file SHA-256s and reader pin inside the artifact; an internal-consistency test recomputing derived fields and verdict from counts (today the committed JSON's verdict could be edited with no test noticing); rekey_signature_definition corrected to state the E→E-conditional baseline the script actually computes (build_crosswave_jobid_check.py:233-247 vs the JSON's per-all-seps shares); and the verdict rule + bands carried into §6 of the design text with a basis — the ratified ruling must contain its own acceptance condition, not point at an artifact for it (:342-348 currently conditions on "its result" without stating what passes).

S4 — Audit-result timing contradiction, and the graded-failure outcome is unstated. #236 :166-167 has runs/e4_e5_audit_v1.json "consumed by the C3 amendment PR" (pre-lock); #230 §12.2 requires only the manifest pre-lock; ADR 0004 §1.5 (:108-113) requires results before the one-shot run. Pin one sequence in both documents. State explicitly in §9: a failed floor invalidates the E4/E5 cells, no candidate can then pass the block (ADR 0004 §1.4), and the audit artifact publishes regardless of result — a designed stop is a graded, publishable outcome, not a re-scoping event.

S5 — v1 floors must be clean-run byte products. The #212 artifacts disclose hand-added metadata post hoc (metadata_edit_note in all three). Acceptable at draft because disclosed; the v1 promotion must meet the M6 byte-reproduction standard (gate_m6_amendment_1...md:33-36): pinned builder → byte-identical artifact, zero hand edits, reproduction test covering the metadata keys.

S6 — The E2 sex×age floor build has unregistered degrees of freedom. :135-139 pins "same method" but not the cell partition (which of A00–A08 gate, sex set, any pooling, thin handling). The one floor built after the draft circulated should run with zero free choices: register the exact cell list and parameters in the draft YAML before the build executes.


NOTES — verified, for the record

N1 — Quoted evidence checks out against the committed artifacts. Recomputed/matched: E4 floors 0.0003–0.0020, 12/12 non-thin; E5 0.011–0.049; E8 0.05–0.20 with the censoring-free 12-month restriction recorded in-artifact; E9 j2j median 0.2264 (floor 0.0766±0.0345), IQR 1.069 (floor 0.0842±0.0522), 545 pairs disclosed as thinnest-gated; E3: 36/63 degenerate quantile-gap cells (recounted per-cell: 12+13+11 across 2020/2022/2024) and 2020 25–34 ECDF 0.018±0.0059; E2 firmsize1 hire full 0.0565±0.0549 vs ex-pandemic 0.0273±0.0211 (36/24 pairs); BDS coarsening keeps 20_99 whole with the exact-merge structure; SUSB flag semantics G/H/J as stated; #214 within-wave means 0.0171/0.0182, seam 0.0945, J2J monthly-equivalents 0.0346–0.0459, and the 0.35%-vs-benchmark naming note carried faithfully into §6; #228's provenance doc confirms the sa tabulation URL, oslp scope, and the five-quarter J2JOD detail window (status flag 11 from 2016Q2), test-pinned (test_j2jod_detail_window); §11's three unit rules match #223 unit_rules and frozen ADR 0003 in substance.

N2 — Evidence-chain hygiene is clean. #212's PR file listing shows stale-base noise (M6 engine/SSA files); its effective diff against master is exactly three floor artifacts + three build scripts — no frozen-contract edits. #228's tier_counts.json 651→662 is the test-count guard incrementing for its 11 added tests. #230 touches one docs file; no sibling gate, no frozen artifact, no gates.yaml edit anywhere in the chain.

N3 — #236 is genuinely pre-outcome where it matters. Sample undrawn; seed 20260717 registered; coders ID-masked and outcome-blind; indeterminates registered conservatively before labels; #235's signature parameters frozen against re-tuning on audit labels; candidate-generation-recall = 1 is correct as a structural property of within-person SIPP attachment. Using #235's measured rates as arm-(b) powering priors is legitimate (power priors do not bias adjudication). §6.1 numeric slots correctly remain referee items.

N4 — Gradedness is mostly in order. E12 deferred with a phase-2 no-go consequence (:382-402) and E11 margins-gated/detail-report-only (:361-371) both apply floors-before-thresholds correctly; #235 carries a REFER_BACK band; no self-rescue paths found beyond the S4 gap. §13 item 9: option (a) is endorsed — (b) would gate on a nine-year-old five-quarter vintage with no temporal floor and is rejected on the record.

§13 dispositions this round (the rest await the revised draft): item 1 — uniform k=4 accepted as policy under S1's honest restatement; per-gate k rejected (threshold-shopping surface); tolerances gated on B5. Item 2 — ex-pandemic basis accepted; full-sample floors stay committed as the on-record alternative (mirrors gate_m6's shock-window partition). Item 3 — decided under B1. Item 4 — ECDF max-gap accepted (degeneracy verified); quantile gaps retained report-only. Item 5 — not accepted as drafted; see B5. Item 6 — Workstream A's permanent-report-only-with-degradation-rule is the right shape; carry the rule verbatim into the draft YAML, not the comment thread. Item 8 — ratifiable conditional on S3's v1 requirements; record the ≤9.4% bound as a footnote, no correction band (consistent with the committed rule). Items 10/13 — confirmed as drafted, with promotion always through the full ceremony (new floor + ADR 0004 audit; no inheritance). Item 14 — SIPP-side 200 now has a power rationale on the record; B-side 10,000 still needs one (B5).

Fix, then this round continues on the draft-block YAML (B2). No candidate work, no gates.yaml motion, and no one-shot registration until the ceremony completes.

— C3 gate-ceremony referee (adversarial round 1)

vahid-ahmadi and others added 3 commits July 19, 2026 16:03
…ew of #235)

Addresses the blocking items in the #235 review. No measured number
changed; no computation altered.

- Withdraw the "pre-registered" claim. Rule and result land in one
  commit (87788eb) with no prior threshold on record, and a first run
  returned a different verdict before the estimator was corrected.
  Relabelled as disclosed re-analysis after a discovered defect.
- Publish both scoring populations. E->E excess is 0.1512
  (PASS_WITH_CORRECTION_BAND); scaled by the E->E share it is 0.0936
  (PASS). Derivable from counts already in the artifact. Which is
  operative is left OPEN for the C3 referee round -- deliberately not
  chosen here, since choosing after seeing both sides of the bar is
  the defect this file documents.
- Relabel gross_id_survival_share as an identity (1 - sep_rate), not
  a bound; drop the dead jobs_kept_share branch.
- Record the 15/30 bands as having no derivation, pending ratification.
- Register known biases: NaN-as-agreement in _rekey_match, unmotivated
  EARN_LOG_TOL, baseline composition mismatch, and the seam-denominator
  circularity in person presence.

Artifact edited to match without re-running (SIPP microdata not on this
machine); edit note records that every added value is recomputable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
All items from the changes-requested review:

- Pre-registration language corrected everywhere to DISCLOSED
  RE-ANALYSIS with the full first-run history in the status field;
  the 15/30 bands and the operative scoring population are marked
  UNRATIFIED referee items.
- Both populations reported with the verdict split visible: E->E
  15.12% (PASS_WITH_CORRECTION_BAND) vs all-separations 9.36%
  (PASS); operative = REFEREE.
- One-sided 95% uppers propagated from binomial SEs (17.65% /
  10.92%) replace the bare point estimate.
- gross_id_survival relabelled as a definitional identity with the
  dead branch removed.
- Strict-NaN sensitivity variant added (missing fields = mismatch):
  excess 12.32% / 7.62% — the NaN-matching bias was real and its
  correction lowers the estimate.
- Composition-mismatch and seam-denominator (person-ID linkage
  under test) caveats recorded.
- Inputs sha256-pinned; artifact-tier test pins the disclosure
  language, both populations, the identity label, uncertainty, the
  strict variant, and the pins.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@daphnehanse11

Copy link
Copy Markdown
Collaborator

Correction to my referee response above, following the #235 review: item Q8 cited the cross-wave check as a pre-registered PASS. That framing was wrong and is retracted — the artifact is a disclosed re-analysis (first-run history now in its status field), its 15/30 bands are unratified, and its verdict is population-conditional: 15.12% on the E→E population (correction band) vs 9.36% on all separations (pass), operative population = referee decision. My substantive position updates to: ratify the #214 seam ruling with the correction band attached unless the referee round adopts the all-separations population and ratifies the 15% bar — in which case clean pass. Either way the ruling itself (J2J for levels, SIPP for structure, seam-aware estimation) stands; what changes is the confidence framing the C3 text should carry. 🤖 Generated with Claude Code

daphnehanse11 and others added 3 commits July 22, 2026 10:02
…on-data re-run, pins

- deployment_scale_note in all three artifacts: the missing
  candidate-context floor is now a RECORDED gap with the root-n
  ratio (~0.71x at full source), for C3 to resolve (accept the
  conservatism, scale analytically, or require ctx floors at v1).
- E8/E9 seeds 5 -> 20 (the E8 cells with sd > mean now have stable
  across-seed sds, e.g. 55-64 any-nonemp floor 0.090 +/- 0.054 vs
  0.141 +/- unstable at 5 seeds); E9 thin flag now counts distinct
  persons per half, not transition pairs (all cells still clear
  200, honestly this time).
- E8 estimand symmetry note: candidates must apply the identical
  full-year-persons restriction.
- All three artifacts REGENERATED on a machine with the staged
  microdata: tenure and E4/E5 measured values reproduce
  byte-identically (confirming the hand-edit disclosure changed no
  numbers, now retired); E8/E9 floors re-estimated under 20 seeds
  with point estimates unchanged.
- Artifact-tier test pins values, statuses, the scale gap, both
  heaping caveats, and the estimand note.

E11/E12 disposition (review's smaller item): Workstream B scope,
recorded in #223's battery; noted in the PR body.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ader pin

Per the #230 round-1 referee review: derived fields and both
verdicts now recompute from the committed counts in an artifact-tier
test (a hand-edited verdict fails); rekey_signature_definition
states the E->E-conditional baseline the script actually computes
(the per-all-seps share fields are marked descriptive); the
sipp_jobs reader commit is pinned in the artifact alongside the
input sha256s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@daphnehanse11

Copy link
Copy Markdown
Collaborator

Workstream A response to the round-1 referee review — the A-side items are closed at new heads (the review's pins predate the #235/#212 review revisions):

S1 (k-precedent misquote): acknowledged and corrected on my side too — my §13 response repeated the "4.2–8" claim; the locked band is 1.8–8 with two ks below 4. My Q1 position is restated on the honest basis: uniform k=4 as policy under the aggregated-sd argument, not as precedent-following.

S3 (#235 v1 mechanics): delivered at the current #235 head — input pu-file SHA-256s and the sipp_jobs reader commit pinned in the artifact; an artifact-tier test now recomputes every derived field and both verdicts from the committed counts (a hand-edited verdict fails CI); rekey_signature_definition corrected to state the E→E-conditional baseline the script computes, with the per-all-seps share fields marked descriptive. The 15/30 bands remain unratified referee items, per the disclosed-re-analysis framing already in the artifact — and I note the referee's item-8 disposition (≤9.4% as footnote, no correction band) is consistent with adopting the all-separations population; no objection.

S4 (audit timing): the contradiction is resolved in #236 — sequence pinned to ADR 0004 §1.5: manifest pre-lock, lock may proceed with the audit designed-but-undrawn, results must exist before the first one-shot run scoring any E4/E5 cell, failed floor invalidates the cells with the artifact publishing regardless (graded stop, per the review's language).

S5 (byte-clean v1 floors): already satisfied at the current #212 head — all three artifacts regenerated on a machine with the staged microdata (tenure and E4/E5 byte-identical to the committed values, confirming the disclosed hand-edits changed no numbers; the edit notes are retired), with an artifact-tier pinning test. The M6 byte-reproduction standard (builder → byte-identical, reproduction test over metadata keys) is accepted for v1 promotion.

B4 (ADR 0004 §3 instantiation for E4/E5): the §9 text is Workstream B's to author, but the A-side substance is on offer: analysis unit = the adjacent-month job-pair (clustering: worker); candidate observables X = age band, sex, industry section, establishment-size code, earnings tercile, multi-job flag, seam-vs-within indicator; weight construction = inverse of a registered logistic adjudication-inclusion propensity over X, trimmed at a registered percentile; operative-version slot = REFEREE per ADR 0004 §6.4. Happy to draft that block into the YAML with @vahid-ahmadi.

S2 (floor scale): the scale gap itself is now RECORDED in all three A-side artifacts (deployment_scale_note, √2 ratio, three resolution options); registering the split machine/fraction/seed in the draft YAML is endorsed and awaited in the B2 re-issue.

🤖 Generated with Claude Code

Workstream A's #230 section-12.2 pre-lock artifact: the complete
ADR 0004 adjudication design for the first-lock scope (SIPP-internal
employer attachment). Registers the frame (the #235 population), the
two arms (accepted-assignment precision; truth-search recall over
the re-key class), scoped stratification with seam oversampling,
REFEREE slots for P_floor/P_design/alpha/power with a worked
binomial example (0.95/0.99/0.05/0.80 -> n=124, c=122 per stratum
before clustering and inflation), the blinded ID-masked coding
protocol with conservative indeterminate handling registered before
labels, provenance including the draw seed (20260717), and the two
leakage freezes (the #235 signature parameters; no label backflow
into readers or hazards). Sample draws only after the referee round
fills the slots.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vahid-ahmadi

Copy link
Copy Markdown
Contributor Author

Addendum for revision 5 / head 1bc161e7ca6a86965708734b425bbc209bed0083: exact #212 head c24809b081c448255a39b74870de70207f615724 is composed through merge commit 00994ddaabad8eb11f7e3999d0730e1a585761a6. The strict-staging, missing raw-input provenance/environment, and missing E9 distinct-person-count blockers are resolved. Every prior #212 measured value reproduced exactly; E9 records stay 16,286 persons and J2J 524. Current artifact, builder, input-sidecar, and measurement-environment hashes are pinned in the control document. Fresh validation: 70 focused tests passed; the full-inventory tier policy passed; collection is 4,275 (803 unit, 1,989 artifact, 804 integration, 520 reproduction, 159 oracle); E11 agreement remains 37/41 and -3.20% to +3.67%; Ruff and diff checks pass. All approval/merge, REFEREE scale/threshold, linkage-QC, draw/blinding, and mechanical YAML blockers remain. This addendum does not approve, ratify, lock, merge, authorize an audit draw, or authorize a candidate run; PR #230 remains draft.

# Conflicts:
#	scripts/first_estimates_birth_evidence.py
#	tests/README-tiers.md
#	tests/estimates/test_birth_evidence_artifact.py
#	tests/tier_counts.json
…adiness-agent

# Conflicts:
#	tests/README-tiers.md
#	tests/tier_counts.json
…mposition-agent

# Conflicts:
#	scripts/first_estimates_birth_evidence.py
#	tests/README-tiers.md
#	tests/estimates/test_birth_evidence_artifact.py
#	tests/tier_counts.json
…mposition-agent

# Conflicts:
#	tests/README-tiers.md
#	tests/tier_counts.json
…mposition-agent

# Conflicts:
#	tests/README-tiers.md
#	tests/tier_counts.json
…mposition-agent

# Conflicts:
#	tests/README-tiers.md
…mposition-agent

# Conflicts:
#	tests/README-tiers.md
#	tests/tier_counts.json
…mposition-agent

# Conflicts:
#	tests/tier_counts.json
@vahid-ahmadi

Copy link
Copy Markdown
Contributor Author

Current-master IC3 composition readiness (173ac15, authored and pushed by vahid-ahmadi):

This remains DRAFT FOR REFEREE — NOT RATIFIED. Composition is not approval, merge, lock, or authorization to run a candidate. Remaining blockers are the refereeable draft YAML + verification, §13 referee decisions, prerequisite approval/merge and linkage-artifact promotion, and the standing no-candidate-before-lock rule. Strong E12 linkage/sorting claims remain no-go without adjudicable linked evidence.

@daphnehanse11 and @MaxGhenis remain the requested reviewers; please continue the cross-workstream/referee review on this exact head when ready. No lock, candidate run, approval, undraft, or merge was performed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants