Skip to content

record(MODEL-TEXT-nemotron-h): the row is blocked on a kernel that landed, and a file glob is why nobody saw it - #1081

Open
localai-bot wants to merge 16 commits into
mainfrom
row/RECORDS-NEMOTRON-H-MATRIX-1074
Open

record(MODEL-TEXT-nemotron-h): the row is blocked on a kernel that landed, and a file glob is why nobody saw it#1081
localai-bot wants to merge 16 commits into
mainfrom
row/RECORDS-NEMOTRON-H-MATRIX-1074

Conversation

@localai-bot

@localai-bot localai-bot commented Aug 16, 2026

Copy link
Copy Markdown
Collaborator

The NemotronH model-matrix row still says the architecture is BLOCKED on
KERNEL-SSM-MAMBA (#496)
because "the Mamba2 SSD core is unported". That kernel landed on 2026-08-13.
Thirty-nine Nemotron commits have landed on main since the row was written,
including the paged forward, the device MoE arm and the ABI driver, and the row
has read INVENTORIED with ☐ required for its spec and none for its
evidence throughout.

FOLLOWING_AGENTS_PROTOCOL

Why this was rewritten rather than merged, and why it was not closed

This branch was cut on 2026-08-16 and main moved 57 commits past it. Main's
version of this row is NEWER than this branch's
— A2-R 598226e96, A2-P
a6df72777, A2-Q2a and the A3 driver c83b96934 all appended to the same cell
after the branch was cut. A three-way merge would have carried this branch's
"the step still REFUSES paged or batched decode because the reached forward is
the HOST reference" over the top of a paged forward that has since landed.

So no keyed record was auto-merged. Every file was taken from main
whole, the tree was asserted byte-identical to it before any edit, and the
scoped edit was written again against what main actually says. The
## Records rule names this shape; the branch's merge commits are squashed
away by squash_merge_commit_message = PR_BODY.

main moved twice more during this run. The third merge (10fe7f475, #1248)
did touch .agents/model-matrix.md, docs/BENCHMARKS.md and docs/STATUS.md,
and Git auto-merged all three. That was verified rather than trusted: the diff
against 10fe7f475 is 13 lines in the matrix and one line each in the two docs,
every one of them this branch's own, with #1248's DSpark row, benchmark row and
status paragraph byte-for-byte equal to main's.

Closing this as superseded was the other candidate and it is wrong, because
the claim it exists to retire is still on main. Verified at 10fe7f475,
.agents/model-matrix.md:285:

Cell On main today Verdict
BLOCKED on KERNEL-SSM-MAMBA (#496), "the Mamba2 SSD core is unported" still there FALSE
non-gated relu² MoE "exists nowhere locally" still there FALSE
ModelOpt MIXED_PRECISION loading "exists nowhere locally" still there FALSE
the DeepSeek-style MTP head "exists nowhere locally" still there TRUE, owed as W5
State INVENTORIED stale
Spike/spec ☐ required stale: five specs are committed
Our code + tests/evidence none stale

docs/FEATURES.md and docs/STATUS.md on main are already current and carry
none of this, so FEATURES is not touched at all and STATUS gains one clause.
The stale record IS the work, which is the case AGENTS.md ## Work happens in a worktree explicitly admits for a record-only pull request.

The blocker, and why a directory listing hides it

#496 landed its host arm at
47960a009 (W1) and its CUDA arm at 43a6c5518 (W2). The kernel is
src/vt/cuda/cuda_mamba2_ssd.cuh — a 692-line header, #included at
cuda_gdn.cu:48 and registered as kMamba2ChunkScan at cuda_gdn.cu:6669, not
a translation unit of its own. A src/vt/*mamba* FILE GLOB therefore returns
nothing and reads as absence, and that glob is the search both the row's text and
#1074 rested on.
nemotron_h.cpp:597 calls vt::Mamba2ChunkScan today, and
nemotron_h.cpp:354 calls vt::MoeRelu2. #496 stays open for its GENERIC
MambaSpec producer, which this architecture does not need, because A1 made the
runner read the model's own KV spec.

Worth carrying: #496's own issue TITLE still asserts "Mamba2 SSD core is
unported"
, so the false claim has a second live copy that this change cannot
reach.

The state, and why it is not ACTIVE

INVENTORIED -> PARTIAL, mark 🚧, with the rollup (INVENTORIED 324->323,
PARTIAL 22->23), the engaged-architecture checklist entry, and the two prose
counts (53->54, 324->323) in the same file.

ACTIVE is refused on a rule. scripts/check-agent-record.py:1257-1266 requires
an ACTIVE row to name a CLAIM-* row that a claim source carries. Measured:
no file under .agents/claims/ (15 of them) names
MODEL-TEXT-nemotron-h-nemotron-hfor-causal-lm, and authoring one for another
session's in-flight work would be a fabricated record. PARTIAL is in
EVIDENCED_STATES, whose contract is exact code and test anchors, which this row
now has.

The A3 gate passed, and my first version of this row said PENDING

I got this one wrong and the correction is instructive, so it stays in the
record.
I searched origin/main and five row/MODEL-NEMOTRON-H-* branches for
a GB10 96/96, found none, and concluded none existed. Then I took
#1221's pull request body as
the current state. Its "Still owed: the sm_121a green-after re-run of the full
96-token gate under this fix" was true when written and stale by the time I read
it. A body is a snapshot of its author's knowledge at write time, not a live
field. No amount of branch searching corrects for that, because the artefact was
never in git.

It is on the NAS, at /usr/local/nas_share/rc/nh1157/. Verbatim from
gate_fixed.out, read directly rather than taken on report:

TOKEN MATCH: 96/96 over 3 prompt(s) (full rows=3, short rows=0, mode=decode)
STRICT PASS

against the pinned oracle vllm=0.23.1rc1.dev1511+g555967922 on
nemotron-3.5-lightning-30b-nvfp4 at revision 29f2d1746d8f41e316523194b19018707749b1b1.

It is the device leg. The binary is libvllm 0.0.3+cuda, and cfg.log
records fp4-mma, cutlass-nvfp4 and cutlass-fp8 as ENABLED for [121a]. The
run logs Asynchronous scheduling is enabled (max_concurrent_batches=2), which
is precisely where device_token_ids is non-null; on the host queue it is always
null and the #1157 defect
cannot arise. The decisive evidence is neither of those: the same binary on the
same checkpoint with only nemotron_h_device.cpp reverted to the fix's
parent scores 4/24 (full rows=0, short rows=3) and bails at 8 generated
tokens. A host-leg run would have been unmoved by that revert. The delta is the
proof; the pass on its own is not.

The pass belongs to #1221, not to main. That branch is
row/MODEL-NEMOTRON-H-ABI-A2P-1157-fix at 6e9e8955, OPEN and CONFLICTING.
main's last touch of nemotron_h_device.cpp is a6df72777 (A2-P). A gate that
passed and a tree that is gated are different facts, and every row here keeps
them apart: the gate PASSES, the merge is owed.

No throughput, latency or memory figure is claimed anywhere in this change.
gate_fixed.out carries 264.4s to load and 327-343s per 32-token prompt. That is
a correctness run whose lm_head and 46 FP8 mamba projections still execute
host-side. Both public rows say "NOT a benchmark" in those words, so nothing can
later lift them into a grid. One config caveat travels with the run:
--gpu-memory-utilization 0.92 did not size the KV pool, which fell back to 256
blocks (#83).

Anchors were re-derived, not carried

The branch's own cited anchor for the MTP deferral, nemotron_h_weights.cpp:989,
now points at a config comment; the deferral is at :1123. Every anchor in the
new cells was re-read against 10fe7f475, and
scripts/check-symbol-anchors.py reports in-repo checked 104 (fresh 104, stale 0), buckets sum 630 vs 630 citations, floor 85.

.agents/issue-index.md

GitHub does not run the merge=union driver .gitattributes:7 sets, so the
union auto-merge was discarded, main's file taken whole, and only the rows
whose KEY main lacks re-appended: #1074 and #1080. main already carries a
#1068 row, so that one is not re-appended. Asserted: main's blob is a
strict BYTE prefix of the result (346246 of 349242 bytes), 369 rows, zero
duplicate keys. This was redone from scratch on each of the three main
merges this branch needed, because a union interleave destroys the prefix
property that makes the file an append-only log.

#1080 is re-measured rather than carried: after this change repairs
nemotron-h-model.md's ## 7. Now to ## Now, 15 specs still write the
numbered spelling, up from the twelve #1080 was filed against. A growing
population is the argument for the checker-semantics fix over a rename sweep, and
both need their own spec and a red-before test. It is listed under ## Owed in
nemotron-h-model.md.

#1217 is deliberately not appended. At 10fe7f475 it is recorded nowhere
in this tree, and both writes ride in the open #1221; appending it here would
give two branches the same append-only key and the union driver would merge them
into a DUPLICATE. It is named under ## Owed in nemotron-h-model.md instead,
where the claim is true whatever #1221 does.

Gates, run bare on this head

agent-preflight.sh --staged 0 (every suite ok, including
test_cpu_x86_llamacpp_floor, which did not hit #618 on this run).
check-agent-record.py 0 (MODEL=377), check-model-checklist.py 0,
check-doc-checkpoint.py --base 10fe7f475 --head HEAD 0,
check-public-doc-tables.py 0, check-supported-models.py 0,
check-issue-index-append-only.py 0, check-symbol-anchors.py 0,
check-now-current.py 0, check-oracle-pins.py 0,
check-commit-style.py --range 0, check-commit-trailers.py --range 0,
check-pr-size.py 0.

Worth naming again: check-doc-checkpoint.py's STATES tuple (:56-65) still
contains neither INVENTORIED nor PARTIAL, so it is structurally blind to this
move and demanded none of the projections. docs/STATUS.md, docs/BENCHMARKS.md
and the spec's ## Now are written because AGENTS.md owes them, not because a
gate asked. The docs/STATUS.md edit first tripped the oversized_cells ratchet
at 45 over 44 and was shortened to fit rather than the ratchet raised.

Inherited, not introduced

windows-msvc-cpu and windows-msvc-vulkan fail on every pull request
(#584,
#968).
test_cpu_x86_llamacpp_floor exits NO_QUIET_WINDOW under box load
(#618).

Closes #1074

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]

mudler added 8 commits August 16, 2026 23:14
…o, and a file glob is why nobody saw it

The NemotronH model-matrix row described `main` as of 2026-08-12 and had not
moved since. It read `INVENTORIED`, blocked on `KERNEL-SSM-MAMBA` (#496),
and named three things that "exist nowhere locally". Every one of those claims
was re-checked against `main` at `10002648199cfbbaf1e423f7c80cacb2f4b56366`
rather than inherited, including the ones the filing issue asserted.

Two of the three now exist. The non-gated relu2 MoE landed at `4d0c399e1`:
`vt::MoeRelu2`, called from `nemotron_h.cpp:354`. ModelOpt `MIXED_PRECISION`
per-module loading landed at `1bc5ef82c`. The third, the MTP head, is still
genuinely owed as W5, and the loader defers its 270 tensors by name.

The blocker itself is the finding. #496 W1 landed the three Mamba2 host
references at `47960a009` and W2 landed the CUDA arm at `43a6c5518`. The
kernel is `src/vt/cuda/cuda_mamba2_ssd.cuh`, a header included by
`cuda_gdn.cu` rather than a translation unit of its own, so a `src/vt/*mamba*`
file glob finds nothing and reads as absence. That is exactly the search this
row's text and the filing issue both rested on. NemotronH calls those ops at
`nemotron_h.cpp:597,620` today. #496 stays open for a GENERIC `MambaSpec`
producer, which this model does not need, because A1 made the runner read the
model's own KV spec.

The row moves `INVENTORIED` to `PARTIAL`, with the rollup, the checklist entry
and the projections that move owes. It does not move to `ACTIVE`, and the
reason is a rule rather than a judgement: `check-agent-record.py` requires an
`ACTIVE` row to name a `CLAIM-*` row that a claim source carries, no claim
record claims this row, and authoring one for another session's in-flight work
would be a fabricated record. `PARTIAL` is what the existing evidence backs,
and it is what the two Qwen3.5 text-only arms carry for the same posture.

Nothing here claims reachability. The forward reached through
`ModelRegistry::Forward` is the host reference, so `nemotron_h_registry.cpp:162`
still refuses paged and batched decode by name. There is no
`examples/nemotron_h_gen`, no ABI token gate, and no throughput, latency or
memory number, and `docs/BENCHMARKS.md` records that as a gap rather than a
number.

Found while doing this: `check-doc-checkpoint.py` matches a spec's live-position
section with `^##\s+Now\s*$`, and thirteen specs write it as `## N. Now`, so a
lifecycle move reds on a section that is present. Filed as #1080. This spec is
the thirteenth and is repaired here, because this change is what makes it the
spec a moving row links. The other twelve are listed under `## Owed`.

Closes #1074

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
…-- issue-index reconciled BY HAND

`origin/main` moved while this reconcile was written. The union driver merged
`.agents/issue-index.md` by interleaving: it put this branch's two rows before
main's `#1066` row, so main's file stopped being a prefix of ours. The index is
an append-only log and the prefix property is what makes it one, so the
auto-merge was discarded and main's file was taken whole, with this branch's
two rows re-appended at the end. Verified: main's blob is a byte prefix of the
result, 290 rows, no duplicate issue number.

Nothing else conflicted. `docs/BENCHMARKS.md`, `docs/FEATURES.md` and
`docs/STATUS.md` were touched on both sides in different places, and the diff
against `origin/main` after the merge is exactly this branch's additions.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
`main` advanced and GitHub reported CONFLICTING on `.agents/issue-index.md`.
GitHub does not run the `merge=union` driver `.gitattributes:7` sets, so the
resolution happens locally where it does.

The union auto-merge of the index was DISCARDED rather than trusted: main's
file taken wholesale, only this row's two rows re-appended, main's file
asserted a strict PREFIX of the result. 293 rows, zero duplicate keys.

`.agents/model-matrix.md` is a KEYED table, not an append-only log, and a union
merge DUPLICATES a keyed row rather than merging it -- that has happened in
this tree before. Verified it was NOT auto-merged (only issue-index.md,
docs/BENCHMARKS.md and docs/FEATURES.md were), and that the NemotronH key
appears exactly ONCE as a table row (`:286`), its other occurrence being the
separate checklist table at `:145`.

`check-agent-record.py` and `check-model-checklist.py` both exit 0.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
`main` advanced (#1084 and #1085 landed) and this branch conflicted for two
different reasons, resolved two different ways.

`.agents/issue-index.md` is append-only under `merge=union`, which GitHub does
not run -- so the union auto-merge was DISCARDED, main's file taken wholesale,
and only this row's own rows re-appended. Main's file is a strict PREFIX of
the result, zero duplicate keys.

`docs/FEATURES.md` was a GENUINE content conflict, not a merge-driver
artifact: both sides added a `NemotronHForCausalLM` row -- this branch's, and
main's from #1085. A keyed table gets ONE row per key carrying BOTH sides'
content, so the two were merged rather than one being chosen:

  - from this branch: the forward computes on landed `vt::Mamba2*` ops (#496)
    -- the finding that retired this row's false blocker -- and the loader's
    quantization breakdown (5935 NVFP4 g16, 46 FP8 W8A8, bf16);
  - from #1085: the A2-Q2a device MoE arm (23 blocks, NVFP4 g16 Marlin) and
    the sharper unreachability statement, G-SAFE refuses FIRST with A2-P
    owning the wiring;
  - from both: no e2e gate and no number.

The merged row is 6 parts with a maximum cell of 214 against the 220 cap, so
it fits without deleting anyone else's entry -- the "cap the entry, never the
file" rule doing its job.

`check-public-doc-tables.py` and `check-agent-record.py` both exit 0, and the
tree carries no conflict markers.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
`main` advanced (#1082 A2-P spec landed) and GitHub reported CONFLICTING on
`.agents/issue-index.md`. GitHub does not run the `merge=union` driver that
`.gitattributes:7` sets, so the resolution happens locally where it does.

The union auto-merge was DISCARDED: main's file taken wholesale, only rows
whose KEY main lacks re-appended, and main's file asserted to be a strict
PREFIX of the result. Zero duplicate keys.

The earlier `docs/FEATURES.md` conflict on this branch -- both sides adding a
`NemotronHForCausalLM` row -- stays resolved as one merged row per key,
carrying the Mamba2-ops finding and the loader breakdown from this branch and
the A2-Q2a device arm and G-SAFE wording from #1085.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
…s row, not merged into it

`main` moved 54 commits ahead of this branch and its NemotronH row is NEWER than
this branch's: A2-R `598226e96`, A2-P `a6df72777`, A2-Q2a and the A3 driver
`c83b96934` all appended to the same cell after this branch was cut. Merging the
two texts would have carried this branch's "the step still REFUSES paged decode
because the reached forward is the HOST reference" over the top of a paged
forward that has since landed.

So no keyed record was auto-merged. `.agents/model-matrix.md`, `docs/FEATURES.md`,
`docs/BENCHMARKS.md`, `docs/STATUS.md` and `.agents/specs/nemotron-h-model.md`
were taken from `b626be75a` WHOLE -- the tree was asserted byte-identical to main
before any edit -- and the scoped edit was then written again against what main
actually says. `.agents/issue-index.md` is append-only under `merge=union`, which
GitHub does not run, so the union auto-merge was discarded, main's file taken
whole, and only the rows whose KEY main lacks re-appended: #1074 and #1080. Main
already carries a #1068 row, so that one is NOT re-appended. Main's blob is a
strict BYTE PREFIX of the result (341815 of 344811 bytes), 366 rows, zero
duplicate keys.

What survives the rewrite is what is still true of main: the row reads
`INVENTORIED`, its `Spike/spec` cell reads `☐ required` against five committed
specs, its evidence cell reads `none`, and it still says the row is BLOCKED on
`KERNEL-SSM-MAMBA` (#496) because "the Mamba2 SSD core is unported". That last
claim is false and re-measured here.

What does NOT survive: this branch's `docs/BENCHMARKS.md` row said the reached
forward is the host reference and named A2-P as owed. Both are stale, and main
already carries a correct `MODEL-NEMOTRON-H-ABI-A2P` row saying the A3 gate is
PENDING. The replacement row is keyed to the MODEL row instead and points at that
one rather than restating it. This branch's `docs/FEATURES.md` edit is dropped
entirely: main's row already describes A2-P, A2-Q2a and G-SAFE correctly, and no
feature surface moves in this change.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
…, and it was re-appended by hand

`origin/main` advanced two commits (#1216, #1237) while this reconcile was being
written. Neither touches `.agents/model-matrix.md`, `docs/STATUS.md`,
`docs/BENCHMARKS.md` or `.agents/specs/nemotron-h-model.md`, so the scoped edit
did not have to be rewritten a second time.

`.agents/issue-index.md` did overlap: main gained three rows and the union driver
interleaved them, putting this branch's two rows before main's new ones and
destroying the prefix property that makes the file an append-only log. GitHub
does not run that driver anyway. So the auto-merge was DISCARDED, main's file
taken whole, and only the rows whose KEY main lacks re-appended: #1074 and #1080.
Asserted: main's blob is a strict BYTE prefix of the result (346246 of 349242
bytes), 369 rows, zero duplicate keys.

The delta against `cdfade6ab` is byte-for-byte the same five files it was against
`b626be75a`.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
…sted

`origin/main` advanced again (#1248) while this branch was being pushed, and this
time it DID touch three of the same files: `.agents/model-matrix.md`,
`docs/BENCHMARKS.md` and `docs/STATUS.md`. Git auto-merged all three, and
AGENTS.md `## Records` says never to accept that for a keyed record.

So the auto-merge was checked rather than trusted. `git diff` against
`10fe7f475` returns 13 changed lines in `.agents/model-matrix.md` and exactly
one each in `docs/BENCHMARKS.md` and `docs/STATUS.md`, and every one of them is
this branch's own scoped edit: the NemotronH row, the rollup, the two prose
counts, the checklist entry, the open-gap row and the status clause. #1248's
`MODEL-SPEC-deepseek-v4-dspark-...` row at `:546`, its DSpark benchmark row and
its DSpark status paragraph are byte-for-byte equal to main's. No unrelated key
moved.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
@localai-bot localai-bot changed the title record(MODEL-TEXT-nemotron-h): the row's blocker landed 23 commits ago, and a file glob is why nobody saw it record(MODEL-TEXT-nemotron-h): the row is blocked on a kernel that landed, and a file glob is why nobody saw it Aug 18, 2026
…e fourth time

`origin/main` advanced twice more (#1245, #1186). Only `.agents/issue-index.md`
overlaps, and it overlaps every time because it is the one file in this change
that every other branch also appends to. That is why GitHub keeps reporting this
pull request CONFLICTING: it does not run the `merge=union` driver
`.gitattributes:7` sets, so the conflict is real on the forge and absent locally.

Resolved the only way that keeps the file an append-only log: the auto-merge was
DISCARDED, main's file taken whole, and only the rows whose KEY main lacks
re-appended -- #1074 and #1080. Asserted: main's blob is a strict BYTE prefix of
the result (346713 of 349709 bytes), 370 rows, zero duplicate keys.

Nothing else overlapped. The delta against `ae581da3e` is the same five files it
was against `10fe7f475`.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
mudler added 2 commits August 18, 2026 16:38
…DING because I read a body written before its own job ran

The previous commit here wrote the A3 end-to-end token gate as `PENDING` and
said no GB10 96/96 existed. That was wrong, and the way it was wrong is worth
keeping: I searched `origin/main` and five `row/MODEL-NEMOTRON-H-*` branches,
found nothing, and then took #1221's PULL REQUEST BODY as the current state. Its
"Still owed: the sm_121a green-after re-run" was TRUE WHEN WRITTEN and stale by
the time I read it. A body is a snapshot of its author's knowledge at write time,
not a live field, and no amount of branch searching corrects for that, because
the artifact was never in git at all.

It is on the NAS. `/usr/local/nas_share/rc/nh1157/gate_fixed.out`:

    TOKEN MATCH: 96/96 over 3 prompt(s) (full rows=3, short rows=0, mode=decode)
    STRICT PASS

against the pinned oracle `vllm=0.23.1rc1.dev1511+g555967922` on
`nemotron-3.5-lightning-30b-nvfp4` at revision `29f2d1746d8f41e316523194b19018707749b1b1`.
Verified by reading the files, not by trusting the report of them.

It is the DEVICE leg. The binary is `libvllm 0.0.3+cuda` and `cfg.log` records
`fp4-mma`, `cutlass-nvfp4` and `cutlass-fp8` `ENABLED for [121a]`. The run logs
`Asynchronous scheduling is enabled (max_concurrent_batches=2)`, which is exactly
where `device_token_ids` is non-null; on the host queue it is always null and the
#1157 defect cannot arise. The decisive one is neither: the same binary on the
same checkpoint with ONLY `nemotron_h_device.cpp` reverted to the fix's parent
scores `4/24 (full rows=0, short rows=3)` and bails at 8 tokens. A host-leg run
would have been unmoved by that revert. The delta is the proof; the pass alone
is not.

So the row now says the gate PASSES and says whose it is. It belongs to #1221,
branch `row/MODEL-NEMOTRON-H-ABI-A2P-1157-fix` at `6e9e8955`, OPEN and
`CONFLICTING`. `main`'s last touch of `nemotron_h_device.cpp` is `a6df72777`
(A2-P), so `main` does not carry the fix and is not gated. A gate that passed and
a tree that is gated are different facts, and this record keeps them apart.

No performance number is claimed anywhere. `gate_fixed.out` carries 264.4s to
load and 327-343s per 32-token prompt; that is a correctness run whose `lm_head`
and 46 FP8 mamba projections still execute host-side. Both public rows say so in
the words "NOT a benchmark", so nothing can later lift them into a grid. One
config caveat also travels: `--gpu-memory-utilization 0.92` did not size the KV
pool, which fell back to 256 blocks (#83).

The row stays `PARTIAL`. `ACTIVE` still fails the `CLAIM-*` rule, and a gate that
passes off `main` is not a reason to move a lifecycle state.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
`main` gained #1221 while this branch was open, so `docs/BENCHMARKS.md` and
`docs/STATUS.md` conflicted on the NemotronH rows. Resolved by taking main's
rows as the base and correcting the one claim that main is now STALE on.

Main says "GB10 read 4/24 ... sm_121a re-run pending a lease" and "sm_121a
re-run pending". That was true when #1221 was written. The re-run has since
happened: the A3 gate PASSES on GB10 at 96/96, `STRICT PASS`, mode=decode,
against `vllm=0.23.1rc1.dev1511+g555967922` on `nemotron-3.5-lightning-30b-nvfp4`
revision `29f2d1746d8f41e316523194b19018707749b1b1`, with a red-before of 4/24
on the SAME binary and checkpoint with only `nemotron_h_device.cpp` reverted.
Artefacts: `/usr/local/nas_share/rc/nh1157/{gate_fixed,gate_red,cfg}.out`.
`cfg.log` shows `fp4-mma`, `cutlass-nvfp4`, `cutlass-fp8`, `marlin-nvfp4` and
`fa2` all `ENABLED for [121a]`, so the build was not degraded, and both legs log
`Asynchronous scheduling is enabled`, which is the path where
`device_token_ids` is live and the defect lived.

The delta, not the pass, is the proof: on the host leg reverting the fix would
have changed nothing.

STILL NOT A BENCHMARK. The 264.4 s load and 43 405 MB peak are recorded as a
correctness run and labelled as such, because NVFP4 `lm_head` (A2-Q2b) and the
46 FP8 W8A8 mamba projections (A2-Q1, #940) still execute host-side. No
throughput, latency or memory ratio is claimed on any axis.

The index was reconciled by discard-and-reapply: main's file taken wholesale,
only rows whose key main lacks re-appended, main's file a strict PREFIX.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
mudler added 5 commits August 19, 2026 11:32
…tted

`main` is green again (#1297 fixed the gcc-16 `getpid` red; the qwen
`build-test-cpu` red had already been fixed by `281b4bc76`), so this branch is
merged forward to drop the stale inherited failures.

`docs/BENCHMARKS.md` was a genuine content conflict: this branch and main
inserted different rows at the same point. BOTH are kept, main's first. No row
dropped, no ratchet raised.

ANCHOR REPAIR. The merge rotted one citation and `check-agent-record.py`
caught it: `stale' 33 > baseline 32`, naming
`nemotron_h_device.cpp:1412 expected NemotronHPagedForward`. Main's count is
exactly 32 with RC=0, and diffing the two report lists isolated the single
extra entry as this row's, so it was repaired rather than the baseline raised.
The symbol now lives at `:1474`.

Worth recording because it nearly shipped: the citation is a markdown link,
and the DISPLAY TEXT and the LINK TARGET are two separate spellings of the
same line number. Repairing only `nemotron_h_device.cpp:1412` left
`[nemotron_h_device.cpp:1474](...cpp#L1412)` -- a citation that READ correct
and POINTED wrong, and which a reader checking the rendered text would have
called fresh. Both halves are now `1474`.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
GitHub does not run the `merge=union` driver `.gitattributes:7` sets, so a
branch touching `.agents/issue-index.md` re-conflicts whenever main appends a
row. Resolved locally where the driver runs: the union auto-merge was
DISCARDED, main's file taken wholesale, only rows whose KEY main lacks
re-appended, and main's file asserted a strict PREFIX with zero duplicate keys.
Any keyed-table collision kept BOTH sides, main's first, so no row is dropped.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
GitHub does not run the `merge=union` driver `.gitattributes:7` sets, so a
branch touching `.agents/issue-index.md` re-conflicts whenever main appends a
row. Resolved locally where the driver runs: the union auto-merge was
DISCARDED, main's file taken wholesale, only rows whose KEY main lacks
re-appended, and main's file asserted a strict PREFIX with zero duplicate keys.
Any keyed-table collision kept BOTH sides, main's first, so no row is dropped.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
GitHub does not run the `merge=union` driver `.gitattributes:7` sets, so a
branch touching `.agents/issue-index.md` re-conflicts whenever main appends a
row. Resolved locally where the driver runs: the union auto-merge was
DISCARDED, main's file taken wholesale, only rows whose KEY main lacks
re-appended, and main's file asserted a strict PREFIX with zero duplicate keys.
Any keyed-table collision kept BOTH sides, main's first, so no row is dropped.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
GitHub does not run the `merge=union` driver `.gitattributes:7` sets, so a
branch touching `.agents/issue-index.md` re-conflicts whenever main appends a
row. Resolved locally where the driver runs: the union auto-merge was
DISCARDED, main's file taken wholesale, only rows whose KEY main lacks
re-appended, and main's file asserted a strict PREFIX with zero duplicate keys.
Any keyed-table collision kept BOTH sides, main's first, so no row is dropped.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

records: the NemotronH model-matrix row still reads INVENTORIED and blocked on #496, 23 commits later

2 participants