Finish #515: per-record E19200 decomposition by published SOI 2.1 AGI-band components

[MaxGhenis]

Why

Issue #515 identified an inconsistent calibration surface: the processed PUF
carried all-zero investment_interest_expense, so the broader
interest_deduction and mortgage-specific HMID controls compiled identical
carrier vectors even though their targets differ. This implements the
adjudicated repair.

Fixes #515

What changed

• Package the TY2015 IRS SOI Table 2.1 rows 10–32 for total interest paid,
  home mortgage interest, deductible points, qualified mortgage-insurance
  premiums, and investment interest.

• Select each record’s published share with source-year E00100 and split
  E19200 deterministically:

  mortgage_i = E19200_i × CH_band / CF_band
  non_mortgage_i = E19200_i − mortgage_i
  

• Preserve the existing raw-E19200 $10M outlier screen and all
  mortgage-only lineage columns.

• Populate investment_interest_expense with the conserving non-mortgage
  residual.

• Align source-year AGI for the processed PUF’s seeded disclosure-record
  replacements, while keeping the Forbes-backed open tail in its known final
  SOI band.

• Switch JCT HMID neutralization from interest_deduction to
  deductible_mortgage_interest.

• Require nonconstant investment-interest signal at release, advance the
  primary QRF checkpoint to schema v4, update Build J source pins, add the
  changelog, and add decomposition/provenance regressions.

The official sources are the
TY2015 Table 2.1 workbook
and Publication 1304, printed pp. 100–101.
The ledger feed has no TY2015 band rows. Every packaged row matches the
official printed table; the repo’s historical SOI asset independently matches
all CF/CH cells. The published band totals are $304.461163B total,
$283.004467B mortgage, and $21.456696B conserving residual.

Allocation and real-input audit

The split is proportional within each half-open published AGI band, so each
record and each band preserve E19200 exactly. It is bit-deterministic and
order-invariant. On the pinned real inputs, 211,677 source-year AGIs align; the
existing outlier screen retains 208,611 donor rows and the weighted identity is
$418.251492B = $378.352506B mortgage + $39.898987B residual.

Checkpoint discipline

This repair runs in the base builder. Its staged resume identity hashes the
builder and all package sources, plus the new TY2015 PUF input’s full SHA; the
inner QRF checkpoint advances to schema v4. It therefore requires and produces
a new base H5. Release target-frame checkpoints and reform-vector caches are
keyed by that H5’s full SHA (and the changed target-registry/reform identities),
so no release materializer or cache-schema bump is warranted. Unlike #557, no
release-time staged transform changes behind an unchanged base hash.

Validation

• uv run pytest packages/populace-build/tests/ -q — rc 0
• Ruff check on all 22 changed Python files — passed
• Ruff format-check on all 22 changed Python files — passed
• Independent implementation audit — clean

Key regressions:

• test_ty2015_e19200_component_rows_are_complete_cited_and_conservative
• test_e19200_donor_split_preserves_each_band_and_published_shares
• test_e19200_donor_split_is_bit_deterministic_and_order_invariant
• test_source_year_puf_agi_matches_pinned_flat_array_artifact
• test_soi_filtered_targets_keep_mortgage_and_broad_interest_distinct
• test_jct_mortgage_reform_neutralizes_only_the_mortgage_component
• test_outer_stage_resume_rejects_changed_builder_code
• test_puf_stage_distinguishes_runtime_prefix_from_artifact_lineage

[MaxGhenis]

Round 2 pushed for both P2 findings: (1) conservation is now bit-exact by reconciliation — mortgage is recomputed as the exact complement of the rounded residual; proven on your 1.53/top-band reproducer and a 20,000-record adversarial sweep (cents-valued totals, full AGI range), with the residual keeping its published-share definition to one rounding; (2) the donor API converts AGI strictly (errors=raise), so NaN/inf reach the explicit finiteness check and nonnumeric raises — regression exercises the puf_tax_unit_donor_from_arrays seam you probed rather than the split function. The P3 coverage note (stubbed engine in the acceptance fixture) stands as-is given your independent live-engine probe confirmed the 90/100/10 behavior. Decomposition + QRF-chain files green locally; full shard on CI.

[MaxGhenis]

Round 3 pushed for the round-2 findings: (1) P2 — non-real dtypes are rejected up front by kind (TypeError) before strict parsing: datetime/timedelta (incl. your NaT-through-the-seam demonstration), complex, and boolean arrays no longer reach pd.to_numeric's finite-sentinel conversions; NaN/inf still fail the explicit finiteness check. (2) P3 regression masking — the AGI test is rebuilt around a positive control: the identical fixture succeeds with valid AGI and empty explicit outputs, so each rejection can only come from the AGI validation, with cause-specific match patterns per class ('finite' vs 'real-valued' vs parse error). Found and fixed en route: my first NaT probe was invalid numpy that raised in the test's own argument expression — both pd.Series([pd.NaT]) and np.asarray(['NaT'], dtype='datetime64[ns]') forms are now exercised. (3) P3 comment — replaced with the proven statement: conservation is exact by Sterbenz's lemma (one of the two subtractions is always exact since either rounded mortgage or rounded residual lands in [total/2, total]), −0.0 reconstructs as +0.0, and the share deviation is bounded at the half-ULP-of-total scale per subtraction (the 'one rounding' at mortgage scale claim is gone — you measured 16–17 mortgage ULPs). Decomposition + QRF-chain files green locally; full shard on CI.

[MaxGhenis]

Round 4 pushed: (1) P2 — the object branch now element-screens via pd.api.types.infer_dtype with a faithful-conversion allowlist (integer/floating/mixed-integer-float/decimal/string/empty): object-wrapped booleans, object-wrapped complex, and categoricals all raise the same real-valued TypeError the typed regressions pin, while Decimal and numeric strings stay accepted per your round-3 note, and missing values still die at the finiteness check. Verified live before writing: pd.Series promotes object-wrapped NaT/Timestamp to datetime64, so those were already caught by the kind gate — bool/complex/categorical were the full residual set. Regressions added for all three object routes plus positive controls for Decimal and numeric-string acceptance. (2) Bad-string assertion now pins the strict parser's own message ('Unable to parse'). (3) Comment says 'bit-for-bit for every accepted total except signed zero'. Decomposition + QRF-chain green locally; CI running.