Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
d10c
force-pushed
the
d10c/learn-inline-expectations-lib
branch
3 times, most recently
from
July 28, 2026 16:03
63edc17 to
52f2ac6
Compare
d10c
marked this pull request as ready for review
July 28, 2026 16:17
owen-mc
reviewed
Aug 14, 2026
d10c
force-pushed
the
d10c/learn-inline-expectations-lib
branch
from
August 20, 2026 09:56
f8bdb41 to
8d9192d
Compare
d10c
force-pushed
the
d10c/learn-inline-expectations-lib
branch
from
September 1, 2026 17:33
9b870e6 to
fc542a0
Compare
d10c
force-pushed
the
d10c/learn-inline-expectations-lib
branch
from
September 8, 2026 18:39
fc542a0 to
4234e9b
Compare
d10c
force-pushed
the
d10c/learn-inline-expectations-lib
branch
from
September 24, 2026 15:50
4234e9b to
7774288
Compare
d10c
force-pushed
the
d10c/learn-inline-expectations-lib
branch
from
September 24, 2026 17:57
7774288 to
ae24f4e
Compare
d10c
force-pushed
the
d10c/learn-inline-expectations-lib
branch
from
September 24, 2026 19:04
ae24f4e to
6b67283
Compare
Contributor
Author
@owen-mc Once the API has stabilized, I can. I already suspect that I'll need to support both |
d10c
force-pushed
the
d10c/learn-inline-expectations-lib
branch
from
September 29, 2026 13:58
6b67283 to
898b86e
Compare
d10c
force-pushed
the
d10c/learn-inline-expectations-lib
branch
from
September 29, 2026 17:25
898b86e to
40ac9c2
Compare
`getAnExpectation` -- the low-level parse of a `// $ ...` expectation comment -- lived as a private predicate inside `Make<Impl>`, reachable only by that module's expectation classes. The upcoming `--learn` postprocessing needs the same parse (to see every expectation on a comment, including ones the running test ignores, so it can preserve them), but it lives in `TestPostProcessing`, outside `Make`. Move the predicate into a new top-level `private module ExpectationParser<Impl>` that both consumers `private import`, so neither has to expose the parse as part of its API nor reach into the other's internals. This is a pure refactor: the predicate body and its callers' behavior are unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1adfa307-c48e-43f5-94e3-8286e4a8c083
The `--learn` support that follows needs to render inline-expectation comments in the syntax of the file being edited, and a single CodeQL database can mix source languages (e.g. Java plus XML), so the marker must be chosen per file rather than per database. Extend `TestPostProcessing::InputSig` with `getStartCommentMarker(relativePath)` (no result -> that file is left untouched by `--learn`) and a defaulted `getEndCommentMarker(relativePath)` (`none()` by default: line-comment languages render no closing marker; block-comment languages can override it later). Each language's `InlineExpectationsTestQuery.ql` implements `getStartCommentMarker`, gated to that language's own source extensions so that other file types (XML/YAML/HTML/ERB/Razor) extracted into the same database are not rewritten with the wrong syntax. This commit only wires the marker API through the signature and the per-language inputs; nothing consumes it yet, so behavior is unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1adfa307-c48e-43f5-94e3-8286e4a8c083
Expose whether each expectation comment uses line-comment syntax so learning can distinguish comments that the current renderer can safely replace. Implement the classification in every inline-expectation adapter, including mixed line/block-comment languages. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1adfa307-c48e-43f5-94e3-8286e4a8c083
`codeql test run --learn` could previously only rewrite `.expected`
files; inline expectations (the `// $ Alert` comments checked by
`InlineExpectationsTest`) had to be fixed by hand. This teaches the
shared test library to compute those source edits so the test runner
can apply them.
The `test-postprocess` query exposes a `learnEdits` relation
(`file, line, operation, startColumn, endColumn, text`) describing the
minimal source rewrite that makes the inline expectations match the
actual query results. Replace edits use 1-based half-open
`[startColumn, endColumn)` ranges. Equal boundaries represent insertion,
and an end column of zero denotes replacement through the end of the
line. Same-line inclusive CodeQL location ends convert to exclusive
boundaries; locations ending on the following line use the to-EOL form.
The relation covers:
- appending a fresh comment carrying every tag learned for a line that
has an unexpected result and no existing comment to merge into;
- rewriting an existing expectation comment as a whole so it matches
the current results: dropping fixed-spurious tags, promoting a
`MISSING:` expectation that now fires, clearing a stale `SPURIOUS:`
annotation, and merging in freshly learned tags, re-rendering the
remaining expectations (or deleting the comment when none remain);
- preserving expectations this test does not own -- e.g. a tag
annotated with a different query's id that shares the source file --
and any trailing regular note (`// $ Alert // note`);
- recording any unexpected result, not just `Alert`.
The comment syntax comes from the `getStartCommentMarker` /
`getEndCommentMarker` markers added to `InputSig` in the previous commit,
so `LearnEditsImpl` renders each edit in the target file's own syntax and
stays language-agnostic. The Actions postprocessor supplies `#` for
`.yml` and `.yaml` sources when implementing the same signature.
Edits are emitted as a query predicate rather than applied here: the
engine consumes `learnEdits` only under `--learn` and ignores it
otherwise, so ordinary `test run` output is unchanged. The supporting
predicates are wrapped in a `private module LearnEditsImpl` so only the
`query predicate learnEdits` is re-exposed.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1adfa307-c48e-43f5-94e3-8286e4a8c083
`getAnExpectation` is a relation that holds between an expectation comment and the parsed parts it carries, not a function returning a single value, so the `get` prefix is misleading and trips the CodeQL predicate-naming style check. Rename it to `hasExpectation`, matching the `has`-prefixed convention for such predicates (and the sibling `hasExpectationWithValue`). Pure rename with no behavior change; reformatted with `codeql query format`. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1adfa307-c48e-43f5-94e3-8286e4a8c083
codeql test run --learn recognises a postprocess query as an inline-expectation test, and so may rewrite the inline `// $ ...` comments in its source, by the `@tags inline-expectation-test` on the query. Add that tag to every language's InlineExpectationsTestQuery.ql so their tests are learnable; without it --learn declines to edit their source and fails, since it cannot group the query with its siblings into a shared-source cohort. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1adfa307-c48e-43f5-94e3-8286e4a8c083
Stop the parsed expectation region at either // or #, matching the syntax documented by the shared parser. This lets --learn preserve notes such as # $ Alert # NOT OK without also interpreting the note words as expectations. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1adfa307-c48e-43f5-94e3-8286e4a8c083
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Teaches the shared inline-expectations test library to describe source edits for
codeql test run --learn. When a tagged inline-expectation postprocessor finds mismatches, it emits alearnEditsrelation that the matching CLI support applies to the annotated source file.Without
--learn, ordinary expectation comparison is unchanged.Rendering learned expectations
The library emits 1-based edits using the engine protocol: appends at the end of a line and half-open replacement ranges
[startColumn, endColumn), withendColumn == 0meaning through end of line.Learning supports line-comment expectations.
ExpectationComment.isLineComment()lets each language adapter identify comments that are safe to rewrite. Existing block comments, including JSX{/* $ Alert */}comments, are left untouched. The append logic also declines to append beside an excluded line comment, where the new text would otherwise become part of that existing comment.The renderer can keep, remove, rewrite, or append standard expectation tags; promote
MISSING:results into expectations; removeSPURIOUS:expectations; and preserve unrelated tags, query qualifiers, and trailing comment text.API and metadata
TestPostProcessing::InputSigsupplies start and end comment markers by source file, leaving room for future languages whose comments require closing delimiters.ExpectationCommentsuppliesisLineComment()across the language adapters.inline-expectation-testtag required by the CLI.Follow-ups
Alert[query1] Alert[query2]..qlrefpostprocessor.