docs: add a regression check against the latest release tag to bug triage

.ai/skills/bug-triage/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: bug-triage
-description: Triage open Comet issues marked `requires-triage` per the project bug triage guide. Classifies each issue as a bug or an enhancement, applies the recommended type (`bug`/`enhancement`), priority, and area labels, removes `requires-triage`, and files a dated summary issue listing what was done. A human reviews the summary issue and closes it when satisfied.
+description: Triage open Comet issues marked `requires-triage` per the project bug triage guide. Classifies each issue as a bug or an enhancement, checks whether each bug is a regression from the most recent release tag, applies the recommended type (`bug`/`enhancement`), priority, area, and `regression` labels, removes `requires-triage`, and files a dated summary issue listing what was done. A human reviews the summary issue and closes it when satisfied.
 ---
 
 <!--
@@ -32,10 +32,13 @@ each one it:
 1. Decides whether the issue is a bug or an enhancement (feature request).
 2. Decides a priority (bugs only) and area labels using the project's triage
    guide.
-3. Applies those labels via `gh` (`bug` or `enhancement`, plus priority/area),
-   ensuring `bug` and `enhancement` are never both present.
-4. Removes the `requires-triage` label.
-5. Records the decision (with rationale) in a single dated summary issue.
+3. Decides whether a bug is a regression from the most recent release, and
+   applies the `regression` label if it is.
+4. Applies those labels via `gh` (`bug` or `enhancement`, plus priority/area
+   and possibly `regression`), ensuring `bug` and `enhancement` are never both
+   present.
+5. Removes the `requires-triage` label.
+6. Records the decision (with rationale) in a single dated summary issue.
 
 `requires-triage` is auto-applied to **every** new issue, not just bug reports,
 so the first decision for each issue is always bug vs. enhancement.
@@ -91,7 +94,7 @@ For each issue, review the title and body and determine:
      be a genuine defect. Classify from the actual content.
    - A bug and an enhancement are mutually exclusive: an issue must never carry
      both `bug` and `enhancement`. If the issue already has the wrong type label,
-     remove it (see Step 5).
+     remove it (see Step 6).
 2. **Priority label** (exactly one, **bugs only**): apply the decision tree from
    the guide.
    - `priority:critical` for correctness issues (silent wrong results, data
@@ -109,7 +112,122 @@ For each issue, review the title and body and determine:
    guide (e.g., a `priority:high` crash that may also produce wrong results),
    note it in the summary.
 
-## Step 4: Skip Issues You Cannot Confidently Classify
+## Step 4: Check Whether Each Bug Is a Regression
+
+Run this step for every issue you classified as a `bug` in Step 3. Skip it for
+enhancements: a missing feature cannot be a regression.
+
+A bug is a **regression** when a workload that behaved correctly on the most
+recent Comet release behaves incorrectly on `main` — wrong results, a new
+failure, a crash, or a fallback-to-Spark path that has since become a wrong
+native answer. It also covers a loss of safety: a query that failed with a
+clear error on the release and now returns silently wrong data is a regression,
+even though it never produced the right answer on either version. A defect that
+already shipped in that release is **not** a regression, however recently it was
+reported.
+
+### Step 4a: Resolve the comparison point from the release tags
+
+Never hard-code a version. Ask GitHub for the current latest release and
+resolve its tag to a commit, so the comparison point moves forward on its own
+as Comet ships:
+
+```bash
+LATEST_RELEASE=$(gh release view \
+  --repo apache/datafusion-comet \
+  --json tagName --jq .tagName)
+git fetch --tags --quiet
+LATEST_RELEASE_SHA=$(git rev-list -n 1 "$LATEST_RELEASE")
+LATEST_RELEASE_DATE=$(git log -1 --format=%cI "$LATEST_RELEASE_SHA")
+```
+
+`gh release view` with no tag argument returns the release GitHub marks as
+"Latest", which excludes pre-releases. Comet marked every `0.x` release as a
+pre-release, so on a repository state where only pre-releases exist this
+returns nothing; in that case fall back to the newest tag by commit date and
+say which tag you used in the summary.
+
+Compare against the **tag's commit date**, not the release's publication date.
+The two differ — Comet's `1.0.0` tag was cut on 2026-08-04 and published on
+2026-08-07 — and commits landing in that window are not in the release.
+
+### Step 4b: Decide, cheapest evidence first
+
+Stop at the first step that gives a definite answer.
+
+1. **Issue creation date.** If the issue was opened before
+   `LATEST_RELEASE_DATE`, the defect was reported before the tag was cut, so it
+   shipped in that release. **Not a regression.** This is free and decisive for
+   most of the backlog.
+2. **Is the defective code present at the tag?** Read the implicated file at
+   the tag and compare it to `main`:
+
+   ```bash
+   git show "$LATEST_RELEASE:native/spark-expr/src/datetime_funcs/unix_timestamp.rs"
+   git grep -n "some_pattern" "$LATEST_RELEASE" -- path/to/dir
+   git diff "$LATEST_RELEASE"..HEAD -- path/to/file
+   ```
+
+   If the defective logic is there verbatim, **not a regression**. If the file
+   or the serde entry that reaches it does not exist at the tag, continue.
+
+3. **Was the path reachable at the tag?** Being absent from the tag is not the
+   same as being a regression. Split it:
+
+   - The feature is new since the release (a new expression serde, a new
+     operator, a new scan mode). A workload that ran on the release cannot
+     reach it. **Not a regression** — it is a defect in new work.
+   - The path existed and was correct, and a post-release change broke it.
+     **Regression.**
+   - The expression previously fell back to Spark (so it was correct) and a
+     post-release change made it run natively with a wrong answer.
+     **Regression** — the user-visible answer changed for the worse.
+
+   A useful check for the reachability question is whether the Scala serde
+   entry, shim, or native registration existed at the tag, not just the Rust
+   kernel:
+
+   ```bash
+   git grep -n "classOf\[SomeExpression\]" "$LATEST_RELEASE" -- 'spark/src/main'
+   git grep -n '"some_function"' "$LATEST_RELEASE" -- native/spark-expr/src/comet_scalar_funcs.rs
+   ```
+
+4. **Bisect or run the reproducer.** If steps 1–3 are inconclusive and the
+   issue has a reproducer, check the tag out into a scratch worktree, build,
+   and run it:
+
+   ```bash
+   git worktree add /tmp/comet-release-check "$LATEST_RELEASE"
+   cd /tmp/comet-release-check/native && cargo build
+   cd /tmp/comet-release-check && ./mvnw test -Dtest=none -Dsuites="<suite>"
+   ```
+
+   Remove the worktree when done (`git worktree remove /tmp/comet-release-check
+--force`). This is the only way to settle a case where the defect depends on
+   a dependency bump (a DataFusion or Arrow/Parquet major version) rather than
+   on Comet's own code.
+
+### Step 4c: Do not trust "pre-existing" in the issue body
+
+Comet issues found during PR review very often say "this is pre-existing, not
+caused by this PR". That claim is about the **pull request under review**, which
+is a narrower and different claim than "this shipped in the last release". A
+defect can be genuinely pre-existing relative to the PR that surfaced it and
+still have landed after the release tag. Verify against the tag either way.
+
+### Step 4d: Apply the label only on positive evidence
+
+- Add `regression` only when step 4b gives you positive evidence that the
+  release behaved correctly.
+- If the evidence is inconclusive, **do not** apply `regression`. Record the
+  issue under "Regression status unclear" in the summary and let the reviewer
+  decide. Do not label the rest of the issue differently on this account —
+  classification, priority, and area still apply.
+- `regression` is orthogonal to priority: a regression keeps the priority its
+  symptoms earn. Per the guide it is also an escalation trigger, so note in the
+  summary when a regression sits below `priority:high`.
+
+## Step 5: Skip Issues You Cannot Confidently Classify
 
 If an issue is too ambiguous to classify with confidence (you cannot tell
 whether it is a bug or an enhancement, or a bug lacks reproduction steps and the
@@ -125,7 +243,11 @@ priority is unclear):
 
 Guessing is worse than skipping.
 
-## Step 5: Apply Labels
+An unclear _regression_ status on its own is not a reason to skip an issue.
+Classify, prioritise, and label it as usual, and record it under "Regression
+status unclear" per Step 4d.
+
+## Step 6: Apply Labels
 
 For each issue you classified in Step 3, apply the labels and remove
 `requires-triage` in a single `gh` call.
@@ -139,6 +261,15 @@ gh issue edit <NUMBER> \
   --remove-label "requires-triage,enhancement"
 ```
 
+For a bug you determined in Step 4 to be a regression, add `regression` too:
+
+```bash
+gh issue edit <NUMBER> \
+  --repo apache/datafusion-comet \
+  --add-label "bug,priority:critical,area:scan,regression" \
+  --remove-label "requires-triage,enhancement"
+```
+
 For an enhancement, add the `enhancement` type label and no priority label:
 
 ```bash
@@ -157,6 +288,10 @@ Notes:
   so it is safe to remove the opposite type unconditionally.
 - Apply a priority label only to bugs. Do not add a priority label to
   enhancements.
+- Apply `regression` only to bugs, and only on the positive evidence described
+  in Step 4d. Never add it to an enhancement.
+- Do not _remove_ an existing `regression` label. If you believe one is wrong,
+  say so in the summary and leave the correction to the reviewer.
 - Pass the labels as a single comma-separated string (no spaces around commas).
 - Quote labels that contain spaces (e.g., `"spark 4"`).
 - Only add labels that already exist in the repo. If a label from the guide is
@@ -168,7 +303,7 @@ If `gh issue edit` fails for any issue, leave that issue's `requires-triage`
 label intact and record the failure in the summary under a "Failed to label"
 section.
 
-## Step 6: File the Summary Issue
+## Step 7: File the Summary Issue
 
 Compute today's date in `YYYY-MM-DD` form (use the system date, not memory):
 
@@ -183,6 +318,8 @@ Body: a markdown report with these sections, in this order:
 1. **Header**
    - Date, total issues processed, count of bugs vs. enhancements, and counts
      per priority
+   - The release tag the regression check compared against, and its commit date
+     (e.g. "Regressions assessed against `1.0.0` (tagged 2026-08-04)")
    - Link to `docs/source/contributor-guide/bug_triage.md`
    - Note that labels have already been applied; the reviewer should spot-check
      and close this issue when satisfied
@@ -198,23 +335,38 @@ Body: a markdown report with these sections, in this order:
 
    - <issue title> ([#1234](https://github.com/apache/datafusion-comet/issues/1234))
      - Area labels: `area:expressions`, `area:scan`
+     - Regression: no — the same code is present at `1.0.0`
      - Rationale: one sentence tying the call to the guide
    ```
 
    The issue number (not the title) is the link target. The title is plain
    text. If there are no area labels, write `Area labels: none`.
 
+   The `Regression:` sub-bullet is required on every bug. Write `yes`, `no`, or
+   `unclear`, followed by the one-line evidence that settled it — which step of
+   Step 4b answered it, and against which tag.
+
 3. **Enhancements** (omit section if empty) — one top-level bullet per issue in
    the same `<title> ([#N](url))` form, with an `Area labels:` sub-bullet and a
    one-sentence rationale for classifying it as an enhancement. Enhancements
-   have no priority subsections.
-4. **Escalations to consider** (omit section if empty) — bullet per issue with
+   have no priority subsections and no `Regression:` sub-bullet.
+4. **Regressions from `<tag>`** (omit section if empty) — every issue you
+   labelled `regression`, collected in one place so a release manager can read
+   them without scanning the priority sections. Bullet per issue in the same
+   `<title> ([#N](url))` form, plus a sub-bullet naming the change that
+   introduced it (a PR or commit, where you identified one) and a sub-bullet
+   with its priority label.
+5. **Regression status unclear** (omit if empty) — bullet per issue with the
+   same `<title> ([#N](url))` form, plus a sub-bullet saying what you checked
+   and what would settle it. These issues were still labelled and had
+   `requires-triage` removed.
+6. **Escalations to consider** (omit section if empty) — bullet per issue with
    the same `<title> ([#N](url))` form, plus a sub-bullet explaining the
    trigger from the guide.
-5. **Skipped — needs more info** (omit if empty) — bullet per issue with the
+7. **Skipped — needs more info** (omit if empty) — bullet per issue with the
    same `<title> ([#N](url))` form, plus a sub-bullet explaining what is
    missing.
-6. **Failed to label** (omit if empty) — bullet per issue with the same
+8. **Failed to label** (omit if empty) — bullet per issue with the same
    `<title> ([#N](url))` form, plus a sub-bullet quoting the `gh` error.
 
 File the issue with `gh`. Use a temp file for the body to keep quoting sane:
@@ -236,8 +388,10 @@ Report back:
 
 1. Number of `requires-triage` issues processed
 2. Counts per priority that were applied
-3. Number skipped (needs more info) and number failed
-4. URL of the new summary issue
+3. The release tag the regression check compared against, how many issues were
+   labelled `regression`, and how many were left unclear
+4. Number skipped (needs more info) and number failed
+5. URL of the new summary issue
 
 Do not paste the full per-issue listing back into the chat; it is in the
 summary issue.
@@ -246,6 +400,11 @@ summary issue.
 
 - Do not invent priority or area labels that are not in the guide
 - Do not create new labels in the repo
+- Do not hard-code a release version in the regression check; always resolve it
+  from the release tags as in Step 4a
+- Do not apply `regression` on the basis of the issue's own "pre-existing"
+  wording, a recent creation date, or a guess — only on the evidence in Step 4b
+- Do not remove a `regression` label that is already on an issue
 - Do not comment on the triaged issues
 - Do not close any triaged issue
 - Do not file the summary issue if there were zero `requires-triage` issues

docs/source/contributor-guide/bug_triage.md

@@ -76,6 +76,65 @@ A bug should be escalated to a higher priority if:
 - A `priority:medium` bug is reported by multiple users or affects a common workload → consider
   escalating to `priority:high`
 - A `priority:low` CI flake is blocking PR merges consistently → escalate to `priority:medium`
+- A bug turns out to be a `regression` from the most recent release → consider escalating one
+  level, because users who upgrade are exposed to it without changing anything on their side
+
+## Regression Label
+
+| Label        | Description                                             |
+| ------------ | ------------------------------------------------------- |
+| `regression` | A bug that did not affect the most recent Comet release |
+
+Apply `regression` to a bug when a workload that behaved correctly on the most recent release
+behaves incorrectly on `main`. That covers wrong results, a new failure, a new crash, and the case
+where an expression used to fall back to Spark (and was therefore correct) and now runs natively
+with a wrong answer. It also covers a loss of safety: a query that failed with a clear error on the
+last release and now returns silently wrong data is a regression, even though it never produced the
+right answer on either version.
+
+A defect that already shipped in the most recent release is **not** a regression, no matter how
+recently it was reported. Neither is a defect in a feature added after that release: a workload
+running on the release cannot reach code that did not exist yet.
+
+`regression` is orthogonal to priority. A regression still gets the priority label its symptoms
+earn, and it is an escalation trigger rather than a priority of its own. It applies only to bugs.
+
+### Determining the Comparison Point
+
+Always compare against the most recent release **tag**, resolved at triage time rather than
+hard-coded, so the comparison point moves forward as Comet ships:
+
+```bash
+LATEST_RELEASE=$(gh release view --repo apache/datafusion-comet --json tagName --jq .tagName)
+git fetch --tags
+git log -1 --format=%cI "$LATEST_RELEASE"
+```
+
+Compare against the **tag's commit date**, not the release's publication date — commits that land
+between the two are not in the release.
+
+### Establishing Regression Status
+
+Work through these in order and stop at the first definite answer:
+
+1. **Issue creation date.** An issue opened before the tag was cut describes behavior that shipped
+   in that release. Not a regression.
+2. **Is the defective code present at the tag?** `git show "$LATEST_RELEASE:<path>"`,
+   `git grep <pattern> "$LATEST_RELEASE"`, or `git diff "$LATEST_RELEASE"..HEAD -- <path>`. If the
+   defective logic is there verbatim, not a regression.
+3. **Was the path reachable at the tag?** Check that the Scala serde entry, shim, or native
+   registration existed, not just the kernel. Code absent from the tag means new work, not a
+   regression — unless a post-release change broke a path that used to be correct.
+4. **Run the reproducer against the tag.** Build the tag in a scratch worktree and run it. This is
+   the only way to settle cases that turn on a dependency bump (a DataFusion or Arrow/Parquet major
+   version) rather than on Comet's own code.
+
+Issues found during PR review often say "this is pre-existing, not caused by this PR". That is a
+claim about the pull request under review, not about the last release; a defect can be pre-existing
+relative to the PR that surfaced it and still have landed after the tag. Verify against the tag.
+
+If the evidence is inconclusive, leave `regression` off and say so on the issue rather than
+guessing.
 
 ## Area Labels
 
@@ -111,9 +170,11 @@ When a new issue is filed:
 3. **Assess correctness impact first.** Ask: "Could this produce wrong results silently?" This
    is more important than whether it crashes.
 4. **Apply a priority label** using the decision tree above (bugs only).
-5. **Apply area labels** to indicate the affected subsystem(s).
-6. **Apply `good first issue`** if the fix is likely straightforward and well-scoped.
-7. **Remove the `requires-triage` label** to indicate triage is complete.
+5. **Check whether the bug is a regression** from the most recent release tag and apply
+   `regression` if it is (bugs only).
+6. **Apply area labels** to indicate the affected subsystem(s).
+7. **Apply `good first issue`** if the fix is likely straightforward and well-scoped.
+8. **Remove the `requires-triage` label** to indicate triage is complete.
 
 ### For Existing Bugs
 
@@ -122,6 +183,9 @@ Periodically review open bugs to ensure priorities are still accurate:
 - Has a `priority:medium` bug been open for a long time with user reports? Consider escalating.
 - Has a `priority:high` bug been fixed by a related change? Close it.
 - Are there clusters of related bugs that should be tracked under an EPIC?
+- Does an open bug need its regression status re-checked against a newer release? A bug that was
+  a regression from one release is still a regression once the next release ships with it
+  unfixed, so `regression` stays until the bug is fixed.
 
 ### Prioritization Principles
 
@@ -183,3 +247,4 @@ Triage is a valuable contribution that doesn't require writing code. You can hel
 - Identifying duplicate issues
 - Linking related issues together
 - Testing whether old bugs have been fixed by recent changes
+- Checking whether an open bug is a `regression` from the most recent release tag