Integrate NSECE child attendance and validate full US population

[hua7450]

Missing child-level attendance inputs can leave CCDF subsidies at zero. This PR adds dataset-side attendance from the real 2024 NSECE household/calendar files, integrates the complete stage into the US fiscal refresh builder, and qualifies it on the pinned BuildP population. PolicyEngine-US defaults remain unchanged.

Implementation

• Verify both ICPSR TSV hashes; derive joint schedules from classified 15-minute ECE calendar blocks, preserving measured nonattendance and excluding K–8 schooling. Handle unpaid and respondent-care gap codes using the documented parent/age distinctions.
• Reconstruct noncalendar schedules using observed regular weekly hours and compatible calendar donors. Days and irregular hours remain explicitly modeled; retain all nearest-hour distance ties.
• Match age, region, parent work, and household income with an explicit age-preserving fallback. Fit sibling dependence on fully observed households. Preserve observed cells, source-person clones, and per-cell provenance.
• Resolve actual ASEC parent pointers and last-week work, map Census regions, and restore temporary income predictors from pinned Census sources using exact person/year/age/line reconciliation. Preserve original raw fields and weights.
• Run the stage after the existing childcare expense producer. Register its source compatibility resource and require all three inputs in the generated release coverage contract. The earlier pool simulation ABI and byte-frozen generation-0 projections retain their existing scope.
• Export a new native H5 through a registered serializer and the shared nullable-boolean boundary, then verify every original entity column, household weight, and time period after reload. --production-stage exercises the same function used by the fiscal builder.

Real population evidence

Reproduction and aggregate reports pin the source, parent, candidate, code, and engine identities. No survey records, donor microdata, or population artifacts are committed.

• 166,321 people / 57,240 households / 31,889 children ages 0–12. All under-13 attendance inputs resolved. The full registered stage and native export completed successfully.
• 51 jurisdictions evaluated: all-zero results fell from 31 to 3 (CA, MD, NV). The three remaining blockers were inspected: California's separate attendance inputs, Maryland's provider type, and Nevada's activity input.
• Weighted under-13 attendance is 48.47%. Exact four-field matching covers 31,152 children; 576 use age/work/income and 161 use age/work.
• Five-fold household-separated source validation: participation 46.51% observed / 46.55% predicted; sibling joint attendance 32.86% / 32.28%, versus 25.99% with independent draws.
• Masked-calendar validation on 1,581 children: weekly hours 13.10 observed / 13.16 reconstructed, and days 1.72 / 1.79. Regular hours remain observed and are preserved.

These are attendance-only counterfactuals using 2026 policies on fixed source ages/incomes, without aging or uprating. Modeled potential benefits are not calibrated spending or caseload estimates. The diagnostics informed development and are not an untouched external acceptance sample.

Scope and remaining limits

The 10,506 donors comprise 7,460 observed calendars and 3,046 reconstructed schedules; 1,105 under-13 source records remain excluded. Conditional matching cannot identify missing schedules for excluded children without assumptions. The May/fall bridge does not observe actual summer attendance or provider-specific pricing.

The explicit export policy retains the existing engine baseline for missing values outside ages 0–12 and labels that provenance. It does not establish nonattendance for older children, including 557 disabled teenagers in the target population. Observed values are preserved, and unresolved under-13 values fail export.

This implements and validates the attendance build stage; it does not publish a replacement population or claim that full population release certification is complete. Reports retain production_ready: false. Licensed sources must be supplied to a release build, and its normal calibration/publication gates still apply.

Survey data for reviewers

Download the NSECE source files used by this PR (Google Drive).

The folder contains NSECE-2024-PR916-source-files.zip (about 5.6 MB compressed; 392 MB extracted):

• 39466-0004-Data.tsv (DS0004): childcare calendar records used to derive attendance days and hours.
• 39466-0005-Data.tsv (DS0005): household and child characteristics, survey weights, and regular childcare hours used for matching and schedule reconstruction.
• README-FILES-USED.txt: explains each file's role, records its SHA-256 checksum, and links this PR.

These are the two unchanged NSECE source files verified against the hashes in the committed validation report. The ZIP does not include CPS ASEC files or the target population. Access: sign in with a PolicyEngine Google account to view and download the ZIP; the folder is not publicly accessible.

Validation

• 916 initial targeted tests passed, covering attendance/source behavior, engine export, architecture, fiscal builder, US bundle generation, and release input coverage.
• 579 CI regression tests passed after correcting the serializer registry, configuration field-coverage pins/report, and pool-spec identity expectation. These runs include every test that failed on the prior CI commit.
• Real source → registered production stage → full pinned population → native H5 reload: passed, with artifact hashes and preserved original values/weights/period.
• All-state before/after comparison and remaining-input diagnostic completed on the same candidate.
• Repository lint, tracked CI test inventory, generated bundle validation, exact field-coverage audit (42,159/42,159 fields), and diff whitespace checks passed.
• Serializer round trips cover native, complete nullable, mixed missing, and all-missing booleans. The corrected writer was rerun on all 166,321 people and preserves the qualified candidate values, dtypes, weights, and period exactly.
• The build wheel was rebuilt successfully and its writer, registry, field ledger, and source recipe were verified against the checked-in sources.
• Full GitHub CI will run on the submitted commit; local tests use synthetic fixtures and do not download licensed surveys.

Addresses #915; provider/activity/older-child gaps remain documented. PolicyEngine/policyengine-us#9405 concerns household aggregation and remains separate.

[hua7450]

Hi @juaristi22, could you please review this PR when you have a chance? This is my first time working in a data repo, and I’m not yet confident that my approach to adding the childcare attendance inputs follows Microcosm’s requirements and conventions.

Please be as critical and thorough as you would normally be—don’t hold back because it’s my first contribution here. I’d especially appreciate feedback on the survey mapping, imputation approach, build integration, and whether the validation is sufficient. Please also point out even minor issues with wording, naming, formatting, or code organization; I want to learn the repo’s standards and get this right.

The PR description includes a “Survey data for reviewers” section with the two source files and a README explaining how they are used. The Drive folder is accessible with a signed-in PolicyEngine Google account. The remaining limitations and the separate integration needed for #893 are documented as well.

Thank you!

[juaristi22]

Program review

Base repository: PolicyEngine/microcosm
PR number: 916
Reviewed head SHA: 3fe3e68
Merge base SHA: a9cc63e
Mode: full
Scope: changed behavior and affected dependencies
Source manifest: /private/tmp/policyengine-command-runs/d374f57aad22/pr-916-review-sources.json
Review status: PARTIAL

Source Documents

No source documents registered; see scope and validation.

What looks good

• The PR addresses the problem at the correct layer: it supplies measured and modeled attendance through the population build instead of changing PolicyEngine-US defaults or inferring attendance from expenses, eligibility, or benefit receipt.
• The survey adapter is deliberately fail-closed. It hash-verifies both NSECE inputs, distinguishes missing/partial/ambiguous calendars from measured nonattendance, excludes K–8 schooling, treats regular and irregular care separately, and derives the three attendance fields jointly from the classified calendar.
• The noncalendar bridge preserves each child's observed regular weekly hours, borrows days and irregular-care intensity jointly, requires compatible participation, retains all ties at the nearest-donor cutoff, uses survey weights, and never feeds reconstructed records back into its donor pool. The masked-calendar assessment holds out entire households, avoiding household leakage.
• The population transfer follows Microcosm's core data contracts well: typed design weights, exact age in every fallback, joint schedule draws, stable source-person identities, clone reconciliation, observed-cell preservation (including zero), and per-cell donor provenance. The sibling mixture is also a meaningful improvement over independent child draws, even though its intensity validation should be expanded as noted below.
• ASEC harmonization uses actual within-household parent pointers and last-week work rather than treating every adult or earner as a parent. Region comes from shared Census constants, and restored income values are protected by file hashes plus person/year/age/line reconciliation while original raw columns remain unchanged.
• Build integration is thoughtfully scoped. The stage runs after the existing childcare-expense producer; all three inputs enter the release-coverage contract; the native exporter writes a new file, reloads it, and verifies the pre-existing entity tables, dtypes, household weights, and time period.
• The validation narrative is unusually candid. It reports source attrition, fallback usage, household-separated diagnostics, all 51 jurisdiction results, and the remaining California/Maryland/Nevada blockers. It consistently labels the result as an attendance-only counterfactual, retains production_ready: false, and does not claim a calibrated release, measured spending, or a change to the default population.

Critical

C1 — CRITICAL (Must Fix): attendance values are not bound to their source receipt, and refreshing an enriched base silently retains stale values (OPEN)

Location: tools/build_us_fiscal_refresh_release.py:1567-1575,9349-9361,11801-11812; packages/microcosm-build/src/microcosm/build/us_runtime/childcare_attendance.py:164-167,208-210; packages/microcosm-build/src/microcosm/build/us_runtime/release_input_coverage.py:567-571,588-612; packages/microcosm-build/src/microcosm/build/us_runtime/l0_refit_export.py:504-523.

Trigger / reproduction: use an otherwise valid base H5 containing non-default values for the three attendance columns (the native candidate produced by this PR is such an H5) and run the fiscal builder without the two NSECE TSV flags. The argument parser only checks that the flags are paired when one is supplied, the stage is skipped when both are absent, the generic H5 loader reads only the six entity tables and discards _childcare_attendance_receipt, and the release-input gate checks only presence/non-default signal. us_source_coverage.json adds childcare evidence only when optional in-memory metadata happens to exist. The build can therefore accept the attendance columns without authenticating the pinned NSECE hashes, seed, matching recipe, modeled-age boundary, or outside-domain policy. A second trigger is to supply new TSVs or a new seed on a previously enriched frame: the imputer labels every pre-existing non-null cell observed, skips every complete source person, then overwrites frame-level metadata with the new source receipt/seed even though the old values were retained.

Expected: a release must either execute the pinned source stage or validate and carry a receipt cryptographically/content-bound to the exact persisted attendance values. Re-running with a changed source identity or seed must either recompute derived cells or reject the incompatible pre-existing provenance.

Observed: arbitrary or stale non-default values satisfy the hard coverage manifest, while source coverage may contain no attendance receipt; when the stage is requested on an enriched input, output values and new metadata can describe different executions.

Impact: this defeats the repository's load-bearing artifact/provenance contract and can certify materially different state childcare-subsidy outputs as if they came from the reviewed NSECE mapping. The committed comparison shows the attendance inputs move potential modeled benefits from about $2.25B to $5.29B, so accepting unbound/stale values is output-material. The new tests cover flag pairing and same-input idempotence, but not receipt-required loading, changed-source/seed refresh, or final source-coverage enforcement.

Should Address

A1 — SHOULD ADDRESS: the final release gate does not reassert row-complete attendance (OPEN)

• Location: packages/microcosm-build/src/microcosm/build/us_runtime/childcare_attendance_stage.py:125-127; packages/microcosm-build/src/microcosm/build/us_runtime/release_input_coverage.py:559-612.
• Trigger: A later build operation, schema conversion, or future refactor introduces null/default attendance in only part of the export population after the NSECE stage has completed.
• Expected: Because the documentation says unresolved under-13 values fail export and the attendance stage's contract requires all three fields to be complete and internally valid, the final pre-write release boundary should re-run that rowwise contract (or have an equivalent attendance-specific gate).
• Observed: with_us_childcare_attendance_inputs calls assert_childcare_attendance_exportable at its own stage boundary, but the final generic input-coverage gate only requires each named column to have at least one finite, non-default observation. A partially null attendance column therefore remains nondegenerate and passes this final gate.
• Impact: The present pipeline is guarded when the stage returns, but the release contract cannot detect partial loss between that point and final serialization. That is a missing boundary check on the precise behavior this PR claims to enforce; a later defect could ship rows that fall back to engine defaults.
• Evidence: The gate documents and implements its column-level criterion at release_input_coverage.py:565-573 and :588-612; the attendance-specific complete-row assertion is only at childcare_attendance_stage.py:127 (and in the standalone native exporter at :169).

A2 — SHOULD ADDRESS: invalid or missing household source identities collapse into one sibling-dependence group (OPEN)

Location: packages/microcosm-build/src/microcosm/build/us_runtime/childcare_population.py:95-100; packages/microcosm-build/src/microcosm/build/us_runtime/childcare_attendance.py:113-115,232-248.

Trigger / reproduction: pass a US frame whose household_source_id column exists but contains a null, or whose person-to-household mapping fails. Harmonization maps the value and immediately calls .astype(str), converting null to the accepted nonempty ID "nan". With fitted sibling_dependence > 0, every affected household uses the same childcare_household_rank hash.

Expected: source household identities used to couple sibling draws should be complete and every person link should resolve; invalid identity must fail closed, as comparable source-ID mapping code in puf_support.py:795-809 does.

Observed: unrelated children with unresolved source identities are treated as one synthetic household for the shared-rank component.

Impact: on malformed or legacy inputs this introduces artificial cross-household dependence and masks an upstream linkage defect. The qualified BuildP parent likely satisfies the late-producer finite-ID invariant, so this does not refute the committed candidate, but the new public stage itself does not enforce its stated identity precondition.

A3 — SHOULD ADDRESS: sibling validation covers only binary participation, while the shared rank couples full schedule intensity (OPEN)

• Location: packages/microcosm-build/src/microcosm/build/us_runtime/nsece_childcare_dependence.py:17-29,32-79; packages/microcosm-build/src/microcosm/build/us_runtime/childcare_attendance.py:171-180,232-256; packages/microcosm-build/src/microcosm/build/us_runtime/nsece_childcare_assessment.py:103-133.
• Trigger: A target household has multiple under-13 siblings and enters the fitted shared-rank branch (rho = 0.7413 in the committed artifact).
• Expected: Validation should cover every material dependence the mechanism induces—at least joint participation, joint days, and joint weekly hours—and show how households with more than two children behave, because attendance days/hours feed subsidy eligibility and amounts.
• Observed: The fit deliberately selects only the youngest two children per fully observed household and estimates one rho from whether both have positive days. The implementation then sorts each donor pool by days and hours and may apply the same household quantile to every sibling, coupling schedule intensity as well as participation and extending the two-child estimate to larger sibships. Cross-validation compares only the both in care probability. The committed aggregate is encouraging for that one target (observed 0.3286, independent 0.2585, fully shared 0.3530), but it does not validate the added hours/days dependence (experiments/us-childcare-attendance/qualified-preparation.json:400-407).
• Impact: Marginal child distributions remain preserved by the quantile construction, but household-level schedule intensity can be too strongly correlated even when the binary joint-attendance moment fits. Nonlinear state eligibility/benefit outcomes can therefore differ without this diagnostic revealing it.
• Evidence: Code trace above and the report's own limitation that sibling assignments still need validation (qualified-preparation.json:393-398). Add held-out weighted joint moments/correlations for days and weekly hours, plus an explicit diagnostic for 3+ child households, before treating the fitted household process as qualified.

Suggestions

S1 — SUGGESTION: foreground the questionnaire-transport discrepancy and define acceptance criteria (OPEN)

• Location: experiments/us-childcare-attendance/qualified-preparation.json:1115-1148; docs/us-childcare-attendance.md:55-68,154-169.
• Trigger: A reviewer reads “qualified” as evidence that the summer/typical-May and new-school-year instruments are exchangeable with the main calendar instrument conditional on the matching cells.
• Expected: The main narrative should state the observed transport discrepancy numerically and define what level/sensitivity would be acceptable for the candidate's intended use.
• Observed: The report estimates weighted regular hours of 12.38 versus a calendar-sample conditional expectation of 14.80 for questionnaire 2, and 10.16 versus 13.92 for questionnaire 3 (about 20% and 37% higher conditional expectations, respectively). The code correctly preserves each noncalendar child's observed regular hours, so those differences do not overwrite that measured field; however, they are evidence that instrument/sample transport is not innocuous for the donor-based days and irregular-care assumptions. The limitations are present and production_ready is false, but no acceptance rule or sensitivity bound translates the discrepancy into a qualification decision.
• Impact: This is not a demonstrated wrong output, but readers can overinterpret the strong masked-calendar mean comparison as validation on the actual noncalendar population. Promote these numbers into the README/docs summary, state that days and irregular care remain unidentified, and report benefit sensitivity to alternative irregular-care/day transport choices.
• Evidence: The assessment itself expressly says the regular-hours comparison cannot validate days or irregular care (packages/microcosm-build/src/microcosm/build/us_runtime/nsece_childcare_assessment.py:163-211) and that the masked test reconstructs cases whose calendars actually exist rather than the other instruments (:215-292).

S2 — SUGGESTION: the persisted operation order collapses three distinct transformations to one generic label (OPEN)

• Location: packages/microcosm-build/src/microcosm/build/us/childcare_attendance_source.json:10-14; packages/microcosm-build/src/microcosm/build/us_runtime/childcare_attendance_stage.py:112-146; committed example at experiments/us-childcare-attendance/qualified-population-comparison.json:32-59.
• Trigger: A release reviewer or diagnostic consumer uses the candidate receipt to reconstruct which source operations ran and in what order.
• Expected: The receipt should retain the distinct declared operations (calendar_attendance, regular_hours_schedule_bridge, joint_weighted_schedule_transfer) and identify the data-fitted sibling-dependence step.
• Observed: operation_order serializes operation.kind, and all three manifest records use the same kind, derive_childcare_inputs. The committed receipt consequently contains that label three times. The fitted sibling_dependence object is separately present, so the numeric fit is not lost, but the structured operation chain is incomplete/ambiguous.
• Impact: This does not change modeled values, but it weakens provenance, makes receipt-level audits harder, and can hide future reordering or insertion of material data-dependent steps.
• Evidence: Serialize the operation-specific operation parameter (ideally the whole normalized operation record) and include fit_sibling_dependence as an explicit material transform.

Coordinator assessment: The code reviewer independently identified the same receipt-auditability issue; it is consolidated here once.

Evidence Gaps

• Required local tests were NOT RUN because the snapshot had no existing environment with pytest/pandas and the review contract prohibited installing dependencies; the exact-head GitHub CI evidence is green.
• The licensed DS4/DS5 TSVs and exact 2024 NSECE value-label codebooks were unavailable, so provider/gap/missingness/respondent-care classifications, file hashes, weight universes, and the NSECE-to-ASEC parent-work crosswalk could not be independently verified.
• The May/fall noncalendar instruments do not observe days or irregular care; the masked-calendar test validates reconstruction on main-instrument records but cannot establish transport validity for the actual noncalendar population.
• The licensed NSECE inputs, pinned ASEC cache, parent H5, and candidate artifacts were unavailable, so the real-data attrition, population enrichment, and 51-jurisdiction comparison could not be independently reproduced.

Notes

• The PR head is 46 commits behind current main; this is informational. Only geography_constants.py changed on both sides, and Git reports the current merge as clean.
• All 23 GitHub checks reported SUCCESS at the exact reviewed head.
• The PR deliberately does not publish a replacement population, change PolicyEngine-US defaults, fix state-specific provider/activity inputs, or resolve the separate household-benefit aggregation issue.

Validation Summary

Inspected the five-commit, 36-file merge-base diff and affected runtime, build, serializer, coverage, documentation, tests, and aggregate evidence. Local tests: NOT RUN (environment unavailable; no dependency install). GitHub CI: 23/23 SUCCESS at head 3fe3e68. Official public source review covered the NSECE study page, Census ASEC variable catalog, and BLS CPI-U table; licensed source bytes/codebooks and full-data artifacts were unavailable.

Timing

setup seconds: 29.00s; scope seconds: 67.00s; parallel review seconds: 765.00s; policy role seconds: 765.00s; code role seconds: 525.00s; adjudication seconds: 0.00s; consolidation cleanup seconds: 140.00s; elapsed seconds: 976.00s

Review Severity

REQUEST_CHANGES. Open findings: 1 critical, 3 should address, 2 suggestions.