Skip to content

🧪 Capture marker scope association in the declarative extraction fixtures - #100

Open
ubmarco wants to merge 7 commits into
mainfrom
feat/scope-association-fixtures
Open

🧪 Capture marker scope association in the declarative extraction fixtures#100
ubmarco wants to merge 7 commits into
mainfrom
feat/scope-association-fixtures

Conversation

@ubmarco

@ubmarco ubmarco commented Aug 19, 2026

Copy link
Copy Markdown
Member

Marker scope association — which declaration a marker attaches to — is computed by
find_associated_scope and flows into the real output, but the shared declarative fixture
harness captured nothing scope-derived, so it had no coverage in the fixture layer at all.

While fixing that, the harness turned out to be snapshotting a test-only projection of
the output rather than the output itself: filepath, remote_url, the full source_map
(columns and end positions), the type discriminator and the marker were all untested,
and only the start row survived as a synthetic line key. So this PR does both: it adds
scope coverage, and it makes the snapshots hold the real production shape.

Deliberately scoped to the existing languages so it can merge independently of the
in-flight TypeScript work; ts/tsx fixtures follow separately.

What changed

The snapshots are now the real output. Each case snapshots exactly what
SourceAnalyse.dump_marked_content() would write — the flat list of Metadata.to_dict()
dicts, sorted by (filepath, source_map.start.row) as production sorts it, built from the
same all_marked_content list production uses. Nested need payload, links as a bare
list, tagged_scope as the plain full-text string, type with its production spelling.

Two deviations, both documented in the harness and the README:

  • filepath is relative to the test root (production emits an absolute path, which differs
    per run and per machine).
  • tagged_scope_type is appended as an additive, test-only key — the associated node's kind
    is not in production output, but it is cheap, deterministic, and the only thing that makes
    "same construct across languages" answerable.

Warnings are a second artefact, not a key in the data. Production writes them to their
own file via dump_warnings() (with a matching load_warnings() for external consumers)
and echoes them to the console — independent of marked_content.json. The fixtures mirror
that separation: a sibling …[warnings].json snapshot per case.

New tests/data/extraction/scope.yaml — 23 cases across cpp, python, csharp, rust, go,
bash and yaml: marker above a function, marker inside a body (the enclosing-scope fallback),
no-scope, the python docstring path (which uses find_enclosing_scope rather than
find_next_scope), and yaml's bespoke structure finder for inline-same-row, leading
key/value and leading list-item association.

jsonc is intentionally not covered: its association goes through a separate bespoke finder
and belongs with a decision about data-format association generally.

Verification

  • tox -e py312-sphinx8-needs5: 368 passed, 1 skipped, 155 snapshots (74 data + 74
    warnings artefacts; case count unchanged by the shape conversion)
  • tox -e docs-clean: no new warnings
  • pre-commit and mypy clean
  • One snapshot was diffed field-by-field against a real dump_marked_content() run on the
    same source: identical keys in production's own order, identical values

What the fidelity increase surfaced

Snapshotting real positions exposed three pre-existing position defects that the reduced
shape could not show. None are fixed here — this PR deliberately only changes tests:

  1. Need-ref columns are shifted. extract_marker computes start_column from the
    pre-.strip() offset but end_column from the post-.strip() length, so every
    need-ref's span is offset by the stripped whitespace. Visible in every need_refs
    snapshot.
  2. Multi-line @rst blocks have a broken source_mapstart.row == end.row, with
    raw offsets into the flattened comment text used as columns. Single-line RST is fine.
  3. Columns are relative to the comment token's text, not the physical line, so an
    indented marker reports a column that is not its column in the file. Only became visible
    once an indented fixture case existed.

Anything consuming source_map for a need-ref, a multi-line RST block, or an indented
marker — source links, "go to code" — is working from wrong positions today.

Follow-up

Every libclang-engine case captures tagged_scope: null, because the preprocessor-aware
path nulls it outright. That looks wrong rather than intentional: libclang decides only
whether a marker is active, not which declaration it belongs to, so those markers should
carry the same scope as the tree-sitter path. Tracked separately — the fixtures now make
the gap visible instead of invisible.

find_associated_scope's result (tagged_scope) flowed into production output
but was invisible to the declarative extraction-fixture harness, which
normalized only needs/need_refs/marked_rst/warnings. Add a "scope" key to
both need and need_ref entries: the associated node's type plus the first
(stripped) line of its text, or null when there is none. Production still
serialises the full node text; the harness only needs enough to prove the
same declaration was selected, without dumping whole function bodies into
expected JSON.
Recapture all existing declarative extraction-fixture snapshots for the new
"scope" key added to need/need_ref entries. Every diff is additive: the
"scope" key only. Reviewed each case's captured scope_type/scope_first_line
by hand; all match the source's expected association (libclang-engine cases
show null, since the analysis explicitly skips scope association for
libclang comments).
Add tests/data/extraction/scope.yaml, pinning find_associated_scope's
outcome across the languages with scope machinery: cpp, python, csharp,
rust, go, bash, yaml. Each language gets a marker directly above a
function/method (binds to it), a marker at the end of a function body
(falls back to the enclosing function/method via find_enclosing_scope), and
a marker with nothing scope-like around it (null). Python additionally
covers a docstring marker (find_enclosing_scope path). YAML additionally
covers its bespoke find_yaml_associated_structure path: an inline same-row
comment, a leading comment bound to a following key-value pair, and one
bound to a following sequence item.

ts/tsx and jsonc are intentionally not covered here.
The declarative extraction snapshots only captured a reduced projection
of the real per-marker payload (id/title/type/links/metadata/line/scope
for needs, need_id/line/scope for refs), leaving filepath, remote_url,
the full source_map (columns and end positions), the MarkedContentType
discriminator, the need-ref marker, and the real tagged_scope text
completely untested.

Extend _normalize() to surface the fields every Metadata subclass
(OneLineNeed/NeedIdRefs/MarkedRst) actually carries: filepath (relative
to tmp_path, since production's absolute path differs per run/machine),
remote_url (deterministically null here — the harness forces
git_remote_url/git_commit_rev to None before run()), the full
source_map, a content_type discriminator (named to avoid colliding with
a need's own "type" field), and scope as {scope_type, scope_text} with
the node's full decoded text exactly as production serializes it
(replacing the truncated {scope_type, scope_first_line} shape). marker
is now included on need_ref entries. The existing needs decomposition
and the line/start_line/end_line keys are kept as-is.

Capturing full source_map fidelity surfaced two pre-existing production
issues, left unfixed here (out of scope): need_id_refs' extract_marker
computes start_column/end_column from the pre-strip() position but the
post-strip() length, so every need-ref's column span is shifted by the
stripped whitespace; and MarkedRst's source_map for multi-line RST
blocks collapses start/end to the same row and uses raw offsets into
the flattened multi-line comment text as "columns", which isn't a real
position for anything past the first line.

No fixture YAML changed; only the harness and the recaptured snapshots.
Update the extraction-fixture README's normalized-contract section to
match the new snapshot shape: filepath, remote_url, source_map,
content_type, and scope (scope_type + full scope_text) on every entry,
plus marker on need_refs. Replace the old "volatile data is omitted"
line with a per-field description and a trimmed exclusion list (only
the tmp_path prefix and the raw comment/node objects are actually
excluded now).
@codecov-commenter

codecov-commenter commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.90909% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.37%. Comparing base (f373b3a) to head (c1542c3).

Files with missing lines Patch % Lines
tests/conftest.py 0.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #100      +/-   ##
==========================================
- Coverage   92.38%   92.37%   -0.01%     
==========================================
  Files          43       43              
  Lines        3743     3740       -3     
  Branches      381      381              
==========================================
- Hits         3458     3455       -3     
  Misses        172      172              
  Partials      113      113              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The extraction-fixture snapshot was a projection invented for the test:
four buckets (needs/need_refs/marked_rst/warnings), a flattened need
payload, links wrapped in a dict, need_id_refs exploded per id, scope
wrapped as {scope_type, scope_text}, and a "content_type" rename.

Rebuild it from what the tool actually writes. _build_marked_content
consumes analyse.all_marked_content — the same flat, (filepath,
source_map.start.row)-sorted list SourceAnalyse.dump_marked_content
dumps — and calls each entry's own Metadata.to_dict(), so the shape and
the ordering come from production, not from re-deriving them out of
oneline_needs/need_id_refs/marked_rst separately. tagged_scope is now
the plain full-text string (or null) exactly as to_dict() serializes
it, need_id_refs carries one need_ids list per marker instead of being
exploded, and the "type" discriminator keeps its real name and value.

Warnings are a second, independent artefact (production never folds
them into the data stream: dump_marked_content and dump_warnings are
separate files), so they get their own snapshot via
snapshot_extraction(name="warnings") instead of a "warnings" key on the
merged object. _build_warnings mirrors
AnalyseProjects.update_warnings()/dump_warnings(): a flat list of
AnalyseWarning.__dict__ records.

Two portability/additive adjustments only: filepath/file_path are
rewritten relative to tmp_path (production's absolute path differs per
run/machine), and each marked-content entry gets one extra key,
tagged_scope_type (the associated node's tree-sitter kind) — not part
of production's output, but free to add and useful for cross-language
comparison.

Recapturing surfaced two pre-existing production quirks, left unfixed
(out of scope): a need-id-reference's source_map columns are shifted by
the width of the whitespace stripped between the marker and its ids
(extract_marker mixes a pre-strip offset with a post-strip length), and
a multi-line rst block's source_map collapses start/end to the same row
with raw string offsets as columns.

No fixture YAML changed; only the harness, conftest's snapshot
extension (now accepting list payloads), and the recaptured snapshots.
Update the extraction-fixture README to match the rewritten harness:
two independent snapshots per case (marked content, warnings) instead
of one merged object; the flat, production-ordered marked-content list
with its real per-type payload (need/need_ids+marker/rst) instead of
the old four-bucket, flattened/exploded projection; the single
tmp_path-relative portability deviation; and the additive, test-only
tagged_scope_type field. Also documents the two known production
column/row quirks the recaptured snapshots now expose as-is.
@ubmarco
ubmarco marked this pull request as ready for review August 20, 2026 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants