🧪 Capture marker scope association in the declarative extraction fixtures - #100
Open
ubmarco wants to merge 7 commits into
Open
🧪 Capture marker scope association in the declarative extraction fixtures#100ubmarco wants to merge 7 commits into
ubmarco wants to merge 7 commits into
Conversation
find_associated_scope's result (tagged_scope) flowed into production output but was invisible to the declarative extraction-fixture harness, which normalized only needs/need_refs/marked_rst/warnings. Add a "scope" key to both need and need_ref entries: the associated node's type plus the first (stripped) line of its text, or null when there is none. Production still serialises the full node text; the harness only needs enough to prove the same declaration was selected, without dumping whole function bodies into expected JSON.
Recapture all existing declarative extraction-fixture snapshots for the new "scope" key added to need/need_ref entries. Every diff is additive: the "scope" key only. Reviewed each case's captured scope_type/scope_first_line by hand; all match the source's expected association (libclang-engine cases show null, since the analysis explicitly skips scope association for libclang comments).
Add tests/data/extraction/scope.yaml, pinning find_associated_scope's outcome across the languages with scope machinery: cpp, python, csharp, rust, go, bash, yaml. Each language gets a marker directly above a function/method (binds to it), a marker at the end of a function body (falls back to the enclosing function/method via find_enclosing_scope), and a marker with nothing scope-like around it (null). Python additionally covers a docstring marker (find_enclosing_scope path). YAML additionally covers its bespoke find_yaml_associated_structure path: an inline same-row comment, a leading comment bound to a following key-value pair, and one bound to a following sequence item. ts/tsx and jsonc are intentionally not covered here.
The declarative extraction snapshots only captured a reduced projection
of the real per-marker payload (id/title/type/links/metadata/line/scope
for needs, need_id/line/scope for refs), leaving filepath, remote_url,
the full source_map (columns and end positions), the MarkedContentType
discriminator, the need-ref marker, and the real tagged_scope text
completely untested.
Extend _normalize() to surface the fields every Metadata subclass
(OneLineNeed/NeedIdRefs/MarkedRst) actually carries: filepath (relative
to tmp_path, since production's absolute path differs per run/machine),
remote_url (deterministically null here — the harness forces
git_remote_url/git_commit_rev to None before run()), the full
source_map, a content_type discriminator (named to avoid colliding with
a need's own "type" field), and scope as {scope_type, scope_text} with
the node's full decoded text exactly as production serializes it
(replacing the truncated {scope_type, scope_first_line} shape). marker
is now included on need_ref entries. The existing needs decomposition
and the line/start_line/end_line keys are kept as-is.
Capturing full source_map fidelity surfaced two pre-existing production
issues, left unfixed here (out of scope): need_id_refs' extract_marker
computes start_column/end_column from the pre-strip() position but the
post-strip() length, so every need-ref's column span is shifted by the
stripped whitespace; and MarkedRst's source_map for multi-line RST
blocks collapses start/end to the same row and uses raw offsets into
the flattened multi-line comment text as "columns", which isn't a real
position for anything past the first line.
No fixture YAML changed; only the harness and the recaptured snapshots.
Update the extraction-fixture README's normalized-contract section to match the new snapshot shape: filepath, remote_url, source_map, content_type, and scope (scope_type + full scope_text) on every entry, plus marker on need_refs. Replace the old "volatile data is omitted" line with a per-field description and a trimmed exclusion list (only the tmp_path prefix and the raw comment/node objects are actually excluded now).
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #100 +/- ##
==========================================
- Coverage 92.38% 92.37% -0.01%
==========================================
Files 43 43
Lines 3743 3740 -3
Branches 381 381
==========================================
- Hits 3458 3455 -3
Misses 172 172
Partials 113 113 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
The extraction-fixture snapshot was a projection invented for the test:
four buckets (needs/need_refs/marked_rst/warnings), a flattened need
payload, links wrapped in a dict, need_id_refs exploded per id, scope
wrapped as {scope_type, scope_text}, and a "content_type" rename.
Rebuild it from what the tool actually writes. _build_marked_content
consumes analyse.all_marked_content — the same flat, (filepath,
source_map.start.row)-sorted list SourceAnalyse.dump_marked_content
dumps — and calls each entry's own Metadata.to_dict(), so the shape and
the ordering come from production, not from re-deriving them out of
oneline_needs/need_id_refs/marked_rst separately. tagged_scope is now
the plain full-text string (or null) exactly as to_dict() serializes
it, need_id_refs carries one need_ids list per marker instead of being
exploded, and the "type" discriminator keeps its real name and value.
Warnings are a second, independent artefact (production never folds
them into the data stream: dump_marked_content and dump_warnings are
separate files), so they get their own snapshot via
snapshot_extraction(name="warnings") instead of a "warnings" key on the
merged object. _build_warnings mirrors
AnalyseProjects.update_warnings()/dump_warnings(): a flat list of
AnalyseWarning.__dict__ records.
Two portability/additive adjustments only: filepath/file_path are
rewritten relative to tmp_path (production's absolute path differs per
run/machine), and each marked-content entry gets one extra key,
tagged_scope_type (the associated node's tree-sitter kind) — not part
of production's output, but free to add and useful for cross-language
comparison.
Recapturing surfaced two pre-existing production quirks, left unfixed
(out of scope): a need-id-reference's source_map columns are shifted by
the width of the whitespace stripped between the marker and its ids
(extract_marker mixes a pre-strip offset with a post-strip length), and
a multi-line rst block's source_map collapses start/end to the same row
with raw string offsets as columns.
No fixture YAML changed; only the harness, conftest's snapshot
extension (now accepting list payloads), and the recaptured snapshots.
Update the extraction-fixture README to match the rewritten harness: two independent snapshots per case (marked content, warnings) instead of one merged object; the flat, production-ordered marked-content list with its real per-type payload (need/need_ids+marker/rst) instead of the old four-bucket, flattened/exploded projection; the single tmp_path-relative portability deviation; and the additive, test-only tagged_scope_type field. Also documents the two known production column/row quirks the recaptured snapshots now expose as-is.
ubmarco
marked this pull request as ready for review
August 20, 2026 13:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Marker scope association — which declaration a marker attaches to — is computed by
find_associated_scopeand flows into the real output, but the shared declarative fixtureharness captured nothing scope-derived, so it had no coverage in the fixture layer at all.
While fixing that, the harness turned out to be snapshotting a test-only projection of
the output rather than the output itself:
filepath,remote_url, the fullsource_map(columns and end positions), the
typediscriminator and themarkerwere all untested,and only the start row survived as a synthetic
linekey. So this PR does both: it addsscope coverage, and it makes the snapshots hold the real production shape.
Deliberately scoped to the existing languages so it can merge independently of the
in-flight TypeScript work;
ts/tsxfixtures follow separately.What changed
The snapshots are now the real output. Each case snapshots exactly what
SourceAnalyse.dump_marked_content()would write — the flat list ofMetadata.to_dict()dicts, sorted by
(filepath, source_map.start.row)as production sorts it, built from thesame
all_marked_contentlist production uses. Nestedneedpayload,linksas a barelist,
tagged_scopeas the plain full-text string,typewith its production spelling.Two deviations, both documented in the harness and the README:
filepathis relative to the test root (production emits an absolute path, which differsper run and per machine).
tagged_scope_typeis appended as an additive, test-only key — the associated node's kindis not in production output, but it is cheap, deterministic, and the only thing that makes
"same construct across languages" answerable.
Warnings are a second artefact, not a key in the data. Production writes them to their
own file via
dump_warnings()(with a matchingload_warnings()for external consumers)and echoes them to the console — independent of
marked_content.json. The fixtures mirrorthat separation: a sibling
…[warnings].jsonsnapshot per case.New
tests/data/extraction/scope.yaml— 23 cases across cpp, python, csharp, rust, go,bash and yaml: marker above a function, marker inside a body (the enclosing-scope fallback),
no-scope, the python docstring path (which uses
find_enclosing_scoperather thanfind_next_scope), and yaml's bespoke structure finder for inline-same-row, leadingkey/value and leading list-item association.
jsoncis intentionally not covered: its association goes through a separate bespoke finderand belongs with a decision about data-format association generally.
Verification
tox -e py312-sphinx8-needs5: 368 passed, 1 skipped, 155 snapshots (74 data + 74warnings artefacts; case count unchanged by the shape conversion)
tox -e docs-clean: no new warningsdump_marked_content()run on thesame source: identical keys in production's own order, identical values
What the fidelity increase surfaced
Snapshotting real positions exposed three pre-existing position defects that the reduced
shape could not show. None are fixed here — this PR deliberately only changes tests:
extract_markercomputesstart_columnfrom thepre-
.strip()offset butend_columnfrom the post-.strip()length, so everyneed-ref's span is offset by the stripped whitespace. Visible in every
need_refssnapshot.
@rstblocks have a brokensource_map—start.row == end.row, withraw offsets into the flattened comment text used as columns. Single-line RST is fine.
indented marker reports a column that is not its column in the file. Only became visible
once an indented fixture case existed.
Anything consuming
source_mapfor a need-ref, a multi-line RST block, or an indentedmarker — source links, "go to code" — is working from wrong positions today.
Follow-up
Every libclang-engine case captures
tagged_scope: null, because the preprocessor-awarepath nulls it outright. That looks wrong rather than intentional: libclang decides only
whether a marker is active, not which declaration it belongs to, so those markers should
carry the same scope as the tree-sitter path. Tracked separately — the fixtures now make
the gap visible instead of invisible.