Skip to content

feat(history): add query insights over the local history store - #2180

Merged
datlechin merged 7 commits into
mainfrom
feat/query-history-insights
Aug 18, 2026
Merged

feat(history): add query insights over the local history store#2180
datlechin merged 7 commits into
mainfrom
feat/query-history-insights

Conversation

@datlechin

@datlechin datlechin commented Aug 18, 2026

Copy link
Copy Markdown
Member

Summary

Adds Query Insights, a Starter-gated tab that summarizes the query history already stored on the user's Mac: most-run shapes, the ones costing the most time, the ones that got slower than the period before, the ones that fail, and an activity chart. Item 5 of #2107.

Everything is computed locally from query_history.db. Nothing leaves the machine.

The part that makes the numbers mean anything

Grouping by raw query text counts WHERE id = 1 and WHERE id = 2 as two different queries, which makes every panel useless: the shapes you actually run hundreds of times each appear once. So this adds SQLQueryFingerprint, which normalizes a statement to its shape the way pg_stat_statements and MySQL's statement digest do, and stores a digest of that shape in a new indexed fingerprint_hash column.

Measured, at the default 10,000-entry retention cap:

cost per refresh
fingerprint on read ~175 ms
stored column, GROUP BY in SQL ~2.7 ms
fingerprint one query at record time 17.5 µs

So the fingerprint is computed once, on the write that already runs in a background Task, and never on the read path. A hash column rather than a normalized-text column: same speed, +100 KB instead of +1.7 MB at 10k rows, and it does not copy query text into a second column, which matters given v3 deliberately dropped parameter_values for that reason.

SQLQueryFingerprint reuses the existing SQLTokenizer and SqlDollarQuote. Building it against the real tokenizer surfaced four defects a naive version would have shipped, each now covered by a test:

  • SELECT $$secret payload$$ … leaked the literal into the shape; the tokenizer has no dollar-quote rule.
  • "..." was treated as a string literal. Outside MySQL it is a quoted identifier, so SELECT "email" FROM "Users" and SELECT "phone" FROM "Orders" collapsed into one group. Double quotes are now identifiers except on MySQL/MariaDB, where they are strings. Guessing wrong that way only splits a group; guessing wrong the other way merges two real tables and reports a wrong count.
  • id=1 and id = 1 produced different shapes.
  • IN (1,2) and IN (1,2,3) produced different shapes.

Two things are deliberately not merged: identifier case (on a case-sensitive server Orders and orders are different tables) and numbers inside names (events_2025 vs events_2026).

Thresholds, and why they are not guesses

A naive "got slower" panel reports noise. Against synthetic history with no real regression in it, a 1.2× threshold flagged 6 of 32 shapes, and raising the sample floor barely moved it. The ratio, not the sample count, is what separates a regression from variance. At 1.5× nothing false survived.

A regression therefore needs all of: 5 runs in both windows, a 1.5× increase, and a 25 ms absolute increase (a query going 1 ms → 2 ms has doubled without costing anything). Only successful runs count, since a query that failed fast is not one that got quicker.

Separately, ranking Slowest by average applies a 3-run floor, or one slow one-off statement sits at the top forever. That is the floor pg_stat_statements users apply by hand when they sort on mean_exec_time.

Other decisions worth flagging

  • Activity buckets go through SQLite's localtime modifier rather than dividing the epoch, so a query run at 22:00 in New York lands on the right day and a daylight-saving change does not shift a bucket. Verified against the system SQLite the app links.
  • Slowest ranks by total time by default (the pg_stat_statements idiom, and the query that actually costs you time), with a segmented control to switch to average. "Slowest" is genuinely ambiguous, so the panel lets the user say which they mean.
  • No outcome filter, on purpose: filtering to succeeded rows would make the failure panel report that nothing ever fails.
  • Date defaults to Last 4 Weeks rather than All Time, because "slower than before" needs a before.

Migration

fingerprint_hash is added and backfilled in a v4 migration. The column is added first and filled second, so an app killed between the two reopens with an unfilled column and finishes the backfill rather than stamping a version that claims work it did not do. The backfill is idempotent and covers both the upgrade-from-v3 and upgrade-from-v2 paths. Three tests cover the upgrade path directly.

Testing

  • verify.sh build: PASS
  • verify.sh test SQLQueryFingerprintTests QueryHistoryInsightsTests QueryHistoryFingerprintMigrationTests QueryHistoryStorageTests QueryHistoryMigrationTests QueryHistoryCaptureTests QueryHistoryEntryTests: PASS
  • verify.sh lint over TablePro plus the three new test files and the UI test: 0 violations, docs references clean
  • 44 new unit tests: fingerprint normalization, the aggregate queries, and the migration
  • QueryInsightsTabUITests covers opening the tab from the Database menu, its singleton behaviour, and that its toolbar identifiers do not collide with the history drawer's

The Pro gate decides how it looks, not what it does

requiresPro applies .disabled plus a scrim and nothing else, so it governs appearance only. The
first version of this view attached .task { await viewModel.activate() } after it unconditionally,
which meant an unlicensed Mac computed every aggregate, held a query-history subscription for the
whole session, and left the resulting numbers in the view hierarchy where the accessibility tree
could read them straight through the blur. Activation is now gated on the same answer the scrim uses,
keyed with .task(id:) so activating a license mid-session still starts it.

QueryInsightsTabUITests covers it: the test sandbox carries no license, so that run sees exactly
what an unlicensed user sees, and it asserts none of the four panels is built.

Before / After

This adds a screen that did not exist, so there is no before. The after is the Query Insights tab: a summary row, an activity chart, and the four ranked panels.

I could not capture a truthful screenshot of it, and I would rather say so than post a misleading one. Three things have to line up at once: the demo/ databases running, Screen Recording granted to whatever drives scripts/export-screenshots.sh, and an active Starter license in the keychain. Without the license the tab renders behind the .requiresPro scrim, so a capture would photograph the upgrade prompt rather than the feature. Without a populated history every panel renders its empty state.

Say the word and I will drive a licensed Debug build against the demo databases and post the real shots here and into docs/images/.

Known gaps

  • The docs screenshots are placeholders. docs/images/query-insights.png and -dark.png are generated stand-ins at the same 1560×960 the other shots use, so the page renders rather than showing a broken image. A real capture needs scripts/export-screenshots.sh with the demo/ databases up, Screen Recording granted, and an active Starter license, since an unlicensed run would photograph the paywall overlay. Please replace before release.
  • Localizable.xcstrings is deliberately not in this PR. The working tree had that file modified by another session (Entra ID strings), and the new strings here will be picked up the next time the catalog is regenerated.

Related

Closes the Query Insights item in #2107. Supersedes the approach in #2111, which grouped by raw query text.

@mintlify

mintlify Bot commented Aug 18, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
TablePro 🟢 Ready View Preview Aug 18, 2026, 6:45 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

…ndow chrome (#2186)

* fix(tabs): draw the editor tab strip as the system tab bar, in the window chrome

Claude-Session: https://claude.ai/code/session_01D5BJ4TrCpMKvVQtQuxwwmH

* fix(tabs): make the whole tab clickable again and stop the strip leaking observers

Claude-Session: https://claude.ai/code/session_01D5BJ4TrCpMKvVQtQuxwwmH
@datlechin
datlechin merged commit 7d5e83d into main Aug 18, 2026
8 checks passed
@datlechin
datlechin deleted the feat/query-history-insights branch August 18, 2026 11:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant