Skip to content

Add weight updates and frame context to graph kernels - #918

Draft
MaxGhenis wants to merge 17 commits into
mainfrom
uk-context-framing-fix-20260913
Draft

MaxGhenis wants to merge 17 commits into
mainfrom
uk-context-framing-fix-20260913

Conversation

@MaxGhenis

Copy link
Copy Markdown
Contributor

The UK full-build graph in #901 needs to normalize weights without changing their kind and reconstruct a declared Frame with its metadata, mass records and column order. This adds WeightUpdate with an ordered-entity receipt and exposes those detached context fields. Original design anchors remain fixed; unrelated sibling mass records cannot leak into a node under an unchanged cache key, and the executor detects mutation of the new context fields.

Independent review findings are resolved, including original-anchor explanation ratios and unambiguous column-order hashing. All 101 selected actual graph tests pass at source 5742c17cef27f02db40997b39d7ac7e8b4f8492e: replay, axis checks, context isolation, original anchors, store and interface contracts. The final isolated run has unchanged source/resource hashes and no unexpected refusals. Verification and limits records the exact evidence; subsequent commit 072c2e88b only updates documentation.

This provides shared contracts for #901. Country-graph integration, native FRS verification and release acceptance remain separate, and the shared calibrate.adam kernel still requires its existing weight-transition contract. Draft pending the full PR matrix and country-owner review.

MaxGhenis and others added 17 commits September 13, 2026 08:32
Source-only verification that WeightUpdate, the ordered-axis weight
receipt, and the three KernelContext frame fields are all absent on
15ebde8, with the exact #901 consumer read that fixes their shape.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
WeightTransition only moves a weight kind forward, so a stage that
recomputes weights it already holds -- a sampling normalization, a
re-solve of an existing calibration -- had no declaration at all, and
the only way to express it was to misdeclare a transition.

WeightUpdate(entity, kind, reason, mass) is that declaration and is
deliberately narrower: incumbent, declared and returned kinds must all
agree; mass is conserve or declared, never free; and reason is
required, non-empty and normative.

Positional replacement values are not self-describing, so a kernel binds
the ordered entity axis it computed them against with
weight_update_receipt, and the executor recomputes that binding from the
incumbent axis. A cache hit reconstructs the KernelResult and re-applies
the REWEIGHT to the current base, so replay re-enters the same check
rather than a parallel rule.

to_kind is a property, not a field, so the two declarations have
disjoint field sets and a WeightUpdate can never canonicalize -- or
serialize -- to the same bytes as a WeightTransition. The transition
payload is byte-for-byte unchanged. Node gains no field, so no existing
node key moves. decl.py is re-locked; kernel.py is untouched.

Raised by the source review of the UK full-build graph (#901, head
051fb97), whose uk.full.normalize node is the first consumer. Its UK
graph stages and calibration science stay in that branch.

Runtime UNTESTED in this lane: source-only ast/ruff and CI test
inventory checks only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ent 26)

The executor projects each entity table in declaration order, so
KernelContext.tables is not the population version's layout, and the
version's Frame metadata and mass log were not reachable from a kernel
at all. A kernel that has to hand its declared projection back to a
legacy function as a Frame could not rebuild one without inventing the
parts it could not see.

KernelContext gains three read-only fields:

- frame_metadata: the version's metadata. The executor passes
  Frame.metadata, which Frame has already deeply frozen; this class adds
  a read-only view and does not itself deep-freeze a mapping built some
  other way. This deliberately differs from the UK branch, which imports
  microcosm.frame.bundle._freeze_metadata into the frozen interface -- a
  frozen contract should not depend on another shard's private name.
- frame_mass_log: the *incoming* Frame mass records. The write side
  already existed (receipt['frame_mass_log_append']); only the read side
  was missing.
- frame_column_order: the version's own column order, restricted to the
  projected columns. An entry that is not exactly an ordering of that
  table's columns is refused, so an order can neither hide a column the
  node was given nor name one it was not -- a column name is itself
  information about the version.

The three ride after artifacts and before tolerances, so amendment 17's
statement that numerics rides at the end stays literally true, and
amendment 19's that artifacts rides before the pair does too; the unit
assertion of adjacency becomes the ordering amendment 19 actually
claimed. Nothing here is normative: Node is untouched and no node key
moves. kernel.py is re-locked; decl.py is untouched.

The acceptance suite's B2 field set follows in its own commit, as
amendment 19's did (a2b6dfb).

Runtime UNTESTED in this lane: source-only ast/ruff and CI test
inventory checks only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Source-only receipt for amendments 25 and 26: exact heads, the absence
proof on 15ebde8, the #901 consumer that fixed the contract shape,
file hashes, the contract decisions that differ from #901, and the
finite invented-only runtime plan left for root to execute.

Runtime UNTESTED in this lane.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The reviewer returned its verdict through the tool result because its
session had no write tool, so the text is filed here unedited rather
than restated. Records the round's scope: F1, F3-F6 accepted, F2 not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The projection handed every node `population.frame.mass_log`. For an
ordinary node that is the version's *cumulative* log, updated after each
member, while its key binds only the version's structural boundary and
the owners of the columns it declared. Adding or re-parameterising an
unrelated sibling therefore changed what a node was shown without moving
its key, so a cache hit could replay output computed against a different
log.

`run_graph` now records each version's log as that version is admitted --
cold execution and a restored hit both reach the same line -- and projects
ordinary nodes from that boundary. A structural node still receives the
cumulative log, because `keys.py` binds its base *and* every ordinary
member of that version through `members`.

The properties distinguish the two cases: an unread same-version appender
is invisible to an ordinary member and moves neither its key nor its
stored bytes, cold-with-sibling equals cold-without and replays across,
while the same appender is visible to the structural boundary whose key
it moves.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`Frame` deeply freezes its metadata and its mass records, but a frozen
dataclass still yields to `object.__setattr__`. Passing the version's own
`_FrozenMapping` leaves and `MassChangeRecord`s by reference therefore made
every kernel -- and anything retaining a context past its own mutation
check -- a live handle on the population: one `object.__setattr__` on a
shared record rewrote the version's log, and unlike a table nothing would
have noticed.

The projection now hands out a deep copy of the metadata and rebuilt mass
records, through the same rule `_observer_snapshot` already followed (now
one shared `_detached_record`). `_context_digest` binds all three fields:
the metadata through the frame format's own store codec, each mass record
field by field, and the projected column order. The codec is guarded, so
a value `Frame` would never have admitted is reported as the mutation it
is rather than raising while the comparison is computed.

The properties cover both halves, and are not vacuous: the rewrite is
asserted to have landed on the retained copy before the live version and
the next node are asserted unchanged. The observer property now rewrites
nested metadata and mass records too -- rewriting tables alone never
touched these fields.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The review's F2 asked for a design-kind `WeightUpdate` to re-anchor
`Population.design_weights`, on the grounds that a later calibration cap
is otherwise "off by k" and that an `EXPAND` afterwards leaves anchors
"mixed". Read against the source, the first is the declared semantics and
the second is not what the code does.

An anchor is the design weight a row entered the population carrying.
`Population.design_weights` is captured once, at CREATE
(`Population.from_frame`), and afterwards only carried by stable entity id
(`_carry_design_weights`), which `patch` hands to `Population.from_frame`
explicitly so the re-derive-from-the-frame default is never taken.
`_carry_design_weights` maps an EXPAND's *copied* rows back to their
source row's original anchor; only a row with no lineage at all reads the
current frame, because it has no earlier weight to be anchored on. So the
values move and no anchor does -- for retained, cloned and entrant rows
alike -- which the new properties assert by running the same EXPAND over
an updated and an un-updated population and comparing the anchors.

`max_weight_ratio` with `weight_anchor='design'` is therefore still
relative to the original design weights, which is what its refusal has
always said ("original design weight"). Re-anchoring would let an
unrelated normalization silently widen every cap declared upstream of it
by that normalization's factor. The review's own T2 case is asserted as
the intended outcome: calibrated weights equal to design weights doubled
by an update are refused at a cap of 1.5 and admitted exactly at 2.0,
reporting a realized ratio of 2.0. The `WeightUpdate` docstring and
amendment 25 now say all of this instead of "ancestry is untouched".

F4: the required-replay property only showed that a hit succeeds. The
stored record's axis binding is now rewritten to the same ids in reverse,
and to a short axis, and each is refused on a run that must hit the
record with no kernel called -- so the check runs on the replay path and
rejects there. A third property re-files the record unchanged and still
replays, so the two refusals are about the binding rather than about
re-filing. The weak test is renamed to the claim it does support.

F5: amendment 25 claimed a re-solve of an existing calibration as a case
it covers. `calibrate.adam@1` emits no `receipt['weight_update']`, so
that declaration would be refused as unverifiable; it is now marked a
future consumer adaptation, in the amendment and in the two docstrings
that repeated it.

F6: the node mass-policy mismatch message said "weight transition's" on a
path both declarations reach.

`decl.py` is re-locked for the amendment-25 text.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ntity

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`_remapped_expand_memberships` aligns clone ordinals: a copied group needs
the same number of copies of every incumbent member, so cloning household
10 without copying persons 1 and 2 would have been refused rather than
proving anything about anchors. The clone now carries its members, which
also lets the property assert the remapped memberships; the entrant
household still joins with none, which is allowed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… plan

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Sep 14, 2026
Merge origin/uk-full-build-graph-registration (051fb97) into the #893
integration head (2300e56) after taking origin/uk-context-framing-fix-20260913
(072c2e8, amendments 25 and 26).

Resolution rules: the graph package, its tests, the charter and the interface
lock are #918's; #901's own draft of WeightUpdate and the frame-context fields,
its test_weight_update.py and its charter section are dropped. US-owned tests
and tools/spec_seed_identity_diagnostics.py are #893's; #901's 10 September
review fixes to them (and the us_spine_stage_contracts.json fixture and the
VerifiedParameterAssets test) are superseded. UK files #901 rewrote into shims
or cut down are #901's. Stage roster is 29: #901's 28 plus frs_relationships
from #903; the retained-leaves / SPI pair stays retired.

Ports of HEAD-side UK changes into #901's modules: #903's frs_relationships
transform in uk_runtime/spine_build.py and its ons_household_type enum-domain
gate artifact through one shared uk_spine_gate_artifacts() used by the
spine-phase gate kernel, the full-build final gates and the driver test; #855's
hierarchy decode in graph_targets.registry_from_payload and on the local
target surface (_surface_records encodes it as TargetRegistry.to_json does,
_local_specs decodes it and keeps it out of metadata), so the schema-8
hierarchy the shared uk_local_target_surface now emits survives the surface
artifact and reaches diagnostics. Invented UK test registries take their
hierarchy from one shared tests/uk_hierarchy_fixtures.py helper.

Dropped run_graph(population_retention="lazy") from full_build_cli: the shared
executor on main has no such mode (see the #893 reconciliation record).

Pins regenerated on the merged tree: calibrate H1 parity, the H2 uk_spine.json
fixture, the UK country-bundle spec digest, the UK release-input coverage
manifest, APPROVED_UV_LOCK_SHA256 / LOCK_SHA256 for the merged uv.lock, and the
microcosm.graph/kernel.py contract in the US implementation inventory.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant