feat(LTX25-IC-LORA): read the IC-LoRA adapter, fuse it into every arm, and stop blaming the metadata (#923)

[localai-bot]

Closes #923. Row LTX25-IC-LORA, spec .agents/specs/ltx25-ic-lora.md. Upstream pin Lightricks/LTX-2 @ fd4ded7f2d88d3da713abcdd4ad41ecc4a9314ca, verified at the local checkout with a clean tree before any anchor was read.

Upstream ICLoraPipeline (ltx-pipelines/ic_lora.py) is video-to-video on the distilled model. This row builds the half of it that was actually missing — the adapter path — and leaves the other half refused, on the causes that genuinely remain.

The reference refusal named a seam that had already landed

This is the finding a fresh review returned FAIL on, and it is the most important thing in this pull request.

The refusal originally blamed the IC-LoRA metadata. That was true, and this row closed it. The row then rewrote the refusal onto the token-APPEND machinery, which was accurate on 2026-08-15 and false on 2026-08-16: row LTX25-TOKEN-APPEND landed that seam in c7cb59fbb while this pull request was open, and the LAST-frame keyframe arm is served on it today. #964 left the wording byte-identical only because #938 was open and its cause was then still true. Both reasons this message had ever given were false at once.

The determination was made again, from the merged tree, rather than inherited. The attention-strength wrapper is not in the way either: on the default arm upstream sets attn_mask = None at conditioning_attention_strength >= 1.0 with no latent mask (iclora_utils.py:159-160) and applies ConditioningItemAttentionStrengthWrapper only if attn_mask is not None (:168-169), so #932 is not the blocker for the default case.

Two causes remain, and the refusal now names both.

1. The reference clip has no pixel path. Upstream reads it at height // scale by width // scale (iclora_utils.py:116-117), refuses a target the factor does not divide (:112-115), keeps frame 0 then every Nth frame (temporal_subsample, :87-89, called at :144) and encodes the whole clip (:145-148). This engine's only pixel-to-latent route encodes exactly one frame at the phase's own resolution and refuses an encode returning more, and nothing anywhere reads ref_video_dir.
2. The reference item is a STAGE-1 item and stage 2 must run UNFUSED. ic_lora.py:108 gives stage 1 loras=tuple(loras) and the reference conditioning (:269-278, :377-402); :119 gives stage 2 loras=() and :314-321 gives it combined_image_conditionings with no reference item. This engine holds ONE DiT, fused at load, that every phase runs.

Serving the arm is filed as #975 rather than done here. Piece 2 changes how the engine holds its weights, not how it conditions; a second resident DiT is ~21 B parameters, which is a memory decision this row cannot take on its own.

The test could not have caught any of that, and now can

The case asserted five substrings of the message. Two were upstream symbol names, present in the pinned checkout whatever this engine can do; three were literals the message declared about itself. None could go red when the engine changed. The reviewer measured it rather than arguing it: replacing the local-cause sentence with a self-declared falsehood while keeping all five substrings left test_ltx2_video at 44 cases / 914 assertions / SUCCESS.

The case now measures the engine first. It renders with and without an appending conditioning item, reads video_tokens — the one trace field written inside the phase loop, so the only one that can observe what the loop does — and requires the grown count to exceed the plain one and both renders to return at the target frame count. Only then does it constrain the message, and the property is positional rather than lexical: every occurrence of a closed cause must sit after the WHAT IS *NOT* THE REASON marker, because recording a ruled-out cause is this message's own convention and has to stay possible.

R1 re-run on the repaired case, at f727cfd85: git diff --stat 1 file, +13/-29; BUILT yes, : error: count 0; exit 1, 49 cases → 1 failed, 1140 assertions → 1 failed (the suite was 49 cases before the #929 merge added one). It fails on REQUIRE(ruled_out != npos) — the restored message has no ruled-out section at all — and would fail again on the positional check.

The bf16 headline was only two thirds gated

The aggregation dtype binds three roundings and only two had cases: B * strength (fuse_loras.py:113) and deltas.add_(weight) (:67-68). The matmul result's own .to(dtype=dtype) had none.

Measured: widening only that one — keeping the f32 accumulator and adding the weight to it before the single store — left test_ltx2_lora at 13/13 and test_ltx2_loader at 31/31.

The new case puts acc = 1 + 2^-8 exactly on a bf16 tie and adds w = 2^-9, so the ported order stores 1.0 and an f32 accumulator stores 1.0078125 — one bf16 step apart in the STORED result, where the final rounding cannot absorb it. Mutation: git diff --stat 1 file, +4/-4; BUILT yes, : error: count 0; exit 1, test_ltx2_lora 14 cases → 1 failed, 2 assertions, reporting 16257 == 16256. test_ltx2_loader stayed 31/31 under the same mutation, and the other 13 lora cases stayed green, which is the measurement that the hole was real.

What lands

• The adapter reader (ltx2_lora.h / .cpp): .lora_A.weight / .lora_B.weight pairs resolved onto the DiT contract through upstream's ComfyUI prefix strip (sd_ops.py:135-137), plus the file's whole __metadata__.
• The fusion: sum((B * strength) @ A) added at load, mirroring fuse_loras.py:99-116.
• Every dtype arm from one hook, placed immediately after MaterializeDitTensor because both quantized branches already return vt::DType::kBF16. On the streaming arm it runs before the device copy, so that arm's "one host buffer live at a time" invariant is unchanged.
• The surface: lora_path / lora_strength load extras and ltx2-gen --lora PATH [STRENGTH]. No ABI change — both ride the existing parallel extras arrays. Load-time rather than per-request, because upstream takes the LoRAs as a DiffusionStage.from_checkpoint constructor argument (ic_lora.py:104-114).

Two deliberate divergences, both argued rather than silent

We do not re-quantize. Upstream's FP8 and NVFP4 rules dequantize, add, and re-quantize (fp8_scaled_mm.py:167-189, nvfp4/fuse.py:13-50) because they keep packed weights resident for their quantized kernels. This tree materializes bf16 on every arm and carries no FP8 or NVFP4 quantizer at all, so there is nothing to re-quantize into. Our fused weight skips upstream's lossy round trip and is slightly more precise on those two arms, at no extra bytes.

An adapter naming a module the contract lacks REFUSES. Upstream skips it (fuse_loras.py:135-137) because its state dict is the whole model. Here the contract is a fixed enumerated set with unported modules already stripped, so a skip would absorb a misnamed key and an inapplicable one alike.

kLoraFusion is retired, not reclassified

It carried DECLARED, NOT REQUESTABLE — an assertion that no request field or load extra asks for LoRA fusion. One now does. #691 predicted this exact drift in its own words and records that the ledger test gates the message text rather than the property. The compiler caught it here, which is weaker than what #691 asks for and does not close #691.

Reachability

Proven on the rendered pixels, not on last_conditioning(): the conditioning trace is filled before the denoise loop, so it cannot see a fused weight, and a first attempt comparing it found every arm identical for that reason rather than because the LoRA did nothing.

Production entry point: vllm_video_engine_load → LoadVideoEngine → Ltx2VideoEngine::Load → Ltx2LoadDitFromSafetensors. The test enters there with a lora_path load extra and compares rendered artifact bytes against an identical request with no adapter.

Mutations

#	mutation	diff	BUILT	exit	result
M1	accumulate the delta in f32 instead of bf16	1 file, +4/-4	yes, no compile_err	1	13 cases → 1 failed, 3 assertions
M2	reachability: delete dit_options.loras.push_back	1 file, +2/-1	yes, no compile_err	1	44 cases → 5 failed, 8 assertions
M3	disable the unknown-target refusal (upstream's skip)	1 file, +1/-1	yes, no compile_err	1	13 cases → 1 failed, 3 assertions
M4	ignore the adapter strength	1 file, +1/-1	yes, no compile_err	1	13 cases → 1 failed, 6 assertions
M5	remove the zero-fusion refusal	1 file, +1/-1	no — -Werror=unused-parameter	NOT_RUN	establishes nothing; redone as M5b
M5b	same, written to compile (fused >= 0)	1 file, +1/-1	yes, no compile_err	1	31 cases → 1 failed, 2 assertions
M6	report the A factor under the B key	1 file, +1/-1	yes, no compile_err	1	13 cases → 1 failed, 2 assertions
R1	restore the pre-repair reference refusal verbatim	1 file, +13/-29	yes, no compile_err	1	49 cases → 1 failed
F3	widen ONLY the matmul-result rounding to f32	1 file, +4/-4	yes, no compile_err	1	test_ltx2_lora 14 → 1 failed, 2 assertions; test_ltx2_loader 31/31 GREEN

R1 and F3 were run at f727cfd85, the commit before the #929 merge. Both carry to the pushed head: git diff f727cfd85 e367026cd is EMPTY over ltx2_lora.cpp and test_ltx2_lora.cpp, and over ltx2_video.cpp it changes only the refusal-counter comment and #929's own additions, leaving the reference Fail(...) string byte-identical. M1 to M6 were run before the review, at the counts the suites had then. Every mutation was restored byte-for-byte, proven by a clean git status. M5 is reported rather than dropped because a mutation that fails to build reads exactly like a passing test.

On porting upstream's tests

There are none to port. Measured at the pin with a positive control so a null result cannot be a wrong search term: find -name 'test_*.py' → 0, find -type d -name 'test*' → 0, conftest.py → 0, grep -rl 'import pytest\|import unittest' → 0, against find -name '*.py' → 280.

Records, and two merges resolved by key

origin/main moved three times during this repair — #966, #939 and #929 — so the branch carries two merges. Both keyed records were resolved by taking main's version and reapplying this row's scoped edit.

• docs/FEATURES.md: 186 of 188 keys byte-identical to origin/main, the two that differ are the two this row owns, and no key is added or removed. Main's LTX-2.5 DiT cell had trimmed IMAGE to IMG and Speed PENDING to PENDING to fit GENkf; that trim is kept and this row's own wording carried the cost. The merged cell is 218 characters against the 220 MAX_CELL_CHARS limit, and no measurement or host qualifier was touched.
• docs/USAGE.md: main's served-last-frame-keyframe paragraph taken whole; the reference paragraph rewritten, because both sides' reasons for that refusal are now false, and the page records that rather than deleting it.
• .agents/issue-index.md: the branch had appended a second #930 row describing that blocker as open. LTX-2.5 token-APPEND: the phase loop is fixed-width, and it is what blocks BOTH the reference arms and the last-frame keyframe #930 is closed, and main already carries the authoritative row, so the duplicate is dropped before it lands — the append-only rule protects rows that exist on main, and this one never did. The #932 row's "blocked behind LTX-2.5 token-APPEND: the phase loop is fixed-width, and it is what blocks BOTH the reference arms and the last-frame keyframe #930" is corrected, and LTX-2.5 reference video: the clip has no pixel path, and stage 2 must run UNFUSED — neither reason the refusal ever gave is still true #975 is appended. issue-index append-only passes in preflight.
• READER ANCHORS: both sides had rewritten the list, so neither survived the merge. Re-derived a third time with the same algorithm test_ltx2_video uses: 779 789 790 852 948 964 966 1044 1069 1174 1215.
• The refusal counter stays at SIX. feat(LTX25-GENERATED-KEYFRAMES): the second feature called "keyframe", refused by what is missing (#920) #929 and this row both rewrote a refusal onto token-append, and c7cb59fbb falsified both while both were open in review. feat(LTX25-GENERATED-KEYFRAMES): the second feature called "keyframe", refused by what is missing (#920) #929 landed leaving the counter at six, reasoning that a near-miss caught in review is not one of them; that reasoning applies here too. The comment records both near-misses rather than claiming a seventh, and a SIX and a SEVEN would have auto-merged into one of them silently.

Gate

HEAD=e367026cd    dirty_files=0
CONFIGURE_EXIT=0
BUILD_EXIT=0      ": error:" count=0
"No space left"=0    "BFD.*assertion"=0
control_no_space=1   control_bfd=1   control_error=1
free disk 42G
ctest -N: Total Tests: 489        CTEST_N_EXIT=0
CTEST_EXIT=0
100% tests passed, 0 tests failed out of 489
loadavg 5.37 at start, 5.36 at end
scripts/agent-preflight.sh: PREFLIGHT_EXIT=0

The three grep counts carry positive controls, so a zero cannot be a broken instrument: the same patterns match their own sample strings and return 1.

scripts/check-doc-checkpoint.py walks per commit (#573) and was run that way, armed with a positive control: --commit b5618b305 exits 1 with the user_usage/docs/USAGE.md message, and both new commits plus the whole origin/main..HEAD range exit 0.

The gate on the pre-#929 tree was also green — 489/489, exit 0 — and is reported only to say that the #929 merge did not change the verdict.

Owed

• LTX-2.5 reference video: the clip has no pixel path, and stage 2 must run UNFUSED — neither reason the refusal ever gave is still true #975 — serving the reference arms: the reference clip's own pixel path, and the stage split that gives stage 2 no adapter. Token-append is no longer part of it.
• LTX-2.5 IC-LoRA: the attention-strength mask arm and N-adapter fusion are both unbuilt, and the second needs a rounding pattern the first does not #932 — the conditioning_attention_strength < 1.0 / conditioning_attention_mask arm, and N-adapter fusion. The latter needs upstream's second rounding pattern (addmm_ with alpha, fuse_loras.py:115), which this row refuses rather than guesses.

GGUF k-quant LoRA fusion is not applicable rather than owed: the LTX-2.5 DiT ships FP8 and NVFP4, and no GGUF LTX DiT exists to fuse into.

Not measured, and not implied

No real-weights IC-LoRA fusion, and no render-quality or speed claim. No GPU was used. The fixture render moves 33 of 91169 artifact bytes, which is a reachability witness on a 2-layer reduced DiT and is not a quality result.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]

[localai-bot]

Operator ruling on the NEEDS_DECISION: #930 dispatches as its own row, not as a tail of this one. You were right to stop rather than lift the refusal.

Three reasons, and the third is the decisive one.

It is a shared seam, and the protocol says so. Token-append blocks at least three arms today, each refused for the same reason by a different row: the last-frame keyframe arm, reference video (VideoConditionByReferenceLatent, which appends — reference_video_cond.py:97-100), and generated keyframe slots (#920, filed independently by a sibling row that hit the identical wall). AGENTS.md: extend a shared seam, never write a parallel path by hand. Three rows queuing behind one fixed-width phase loop is exactly the shape that wants one row.

Two agents found it independently, which is evidence it is real. #920 and #930 were filed hours apart by rows that never spoke, both naming the phase loop and Ltx2LatentState. That is a structural gap, not a local awkwardness.

Lifting the refusal here would have shipped a wrong render, and you caught that the refusal named only the first of two causes. Closing the adapter blocker and then rewriting the refusal onto the cause that actually remains is the correct outcome for this row — the refusal is now true, which is worth more than a feature that is quietly false.

Two things from your report I want recorded rather than lost.

M5 not building, reported as NOT_RUN establishing nothing, is exactly right. A mutation that fails to compile reads as a passing test, and the honest move is the one you made — re-run it compiling as M5b rather than bank the green.

Proving reachability on rendered pixels rather than last_conditioning() is the sharper method, and your reason is the useful part: the trace is filled before denoise, so it cannot observe a fused weight, and your first attempt found every arm identical for that reason. That belongs in the spec, because the next person will reach for the trace first.

Also noted: another agent overwrote your PR-body file in the shared scratchpad. That is the third occurrence today; the scratchpad is contended and nothing generically named is safe there. And thank you for flagging the pkill -9 ninja — the box is shared, and killing by PID after checking /proc/<pid>/cwd is the right correction.

FOLLOWING_AGENTS_PROTOCOL