From 05d5941e59016cf4741984c8df91625a8f2000d4 Mon Sep 17 00:00:00 2001 From: Andrey Listopadov Date: Tue, 28 Apr 2026 13:50:52 +0300 Subject: [PATCH 1/6] Document $psql / $psql-cancel and SQL Console v3 --- docs/api/rest-api/other/sql-endpoints.md | 75 +++++++++++++++++++++++- docs/overview/aidbox-ui/README.md | 9 +++ docs/overview/release-notes.md | 13 ++++ 3 files changed, 96 insertions(+), 1 deletion(-) diff --git a/docs/api/rest-api/other/sql-endpoints.md b/docs/api/rest-api/other/sql-endpoints.md index 32a894563..e16e8b6ec 100644 --- a/docs/api/rest-api/other/sql-endpoints.md +++ b/docs/api/rest-api/other/sql-endpoints.md @@ -1,5 +1,5 @@ --- -description: Execute raw SQL queries directly in Aidbox using $sql endpoint with JDBC parameter support. +description: Execute raw SQL queries directly in Aidbox via $sql, $psql, and $psql-cancel endpoints. --- # SQL endpoints @@ -39,6 +39,79 @@ POST /$sql?_format=yaml {% endtab %} {% endtabs %} +## $psql + +Run a raw multi-statement SQL script in a single request. The endpoint is what the Aidbox UI SQL Console uses; `/$notebook-psql` is an alias with identical behavior. + +Request body: + +```yaml +POST /$psql + +{ "query": "SELECT 1; SELECT 2", "limit": 1000 } +``` + +- `query` — the SQL text. Sent verbatim to PostgreSQL; Aidbox does not split, trim, or rewrite it. +- `limit` (optional) — applied via JDBC `setMaxRows` to cap each result set. + +Response (success): + +```yaml +{ "status": "success", + "duration": 12, + "query": "SELECT 1; SELECT 2", + "result": [ + { "type": "rset", "data": [{ "?column?": 1 }] }, + { "type": "rset", "data": [{ "?column?": 2 }] } + ] } +``` + +`result` is an array of one entry per statement. `:type` is `rset` for queries that return rows (`SELECT`, `INSERT … RETURNING`, `EXPLAIN`, …) and `count` for statements that report a row count (`UPDATE`, `DELETE`, `INSERT` without `RETURNING`). + +Response (error): + +```yaml +{ "status": "error", + "error": "ERROR: syntax error at or near \"SELEC\"\n Position: 1", + "duration": 4, + "query": "SELEC 1", + "position": 1 } +``` + +### Execution headers + +Every header below is optional. Defaults match a single-transaction read-write run. + +| Header | Value | Effect | +|---|---|---| +| `X-Aidbox-Sql-Autocommit` | `true` | Run outside a transaction. Required for `VACUUM`, `CREATE INDEX CONCURRENTLY`, `REINDEX CONCURRENTLY`. Rejected when [`db.pass-auth-vars`](../../../reference/all-settings.md#db.pass-auth-vars) is on **and** the request carries a resolvable identity — autocommit would drop the SQL identity injection that RLS relies on. | +| `X-Aidbox-Sql-Timeout` | seconds, 1..86400 | Per-query `statement_timeout`. Empty / non-numeric / negative / out-of-range values are ignored. | +| `X-Aidbox-Sql-Read-Only` | `true` | Run as read-only. Writes raise `ERROR: cannot execute … in a read-only transaction`. | +| `X-Aidbox-Sql-Query-Id` | UUID | Tags the PG session via `application_name = aidbox-psql:`. The same UUID is used to cancel via `/$psql-cancel`. | +| `X-Aidbox-Sql-Async` | `true` | Fire-and-forget background execution. Returns `202 { "operation-id": "" }` immediately. The query runs server-side; result rows are not retained — only `status` / `duration` / `query` / `error` are kept in `db_scheduler.scheduled_tasks_history`. The same handler accepts the operation-id as a `query-id` for cancellation. | + +### Breaking change in 2604 + +Prior versions returned a vector of per-statement debug objects and accepted an `execute=true` query parameter to switch between two execution paths; multi-statement scripts were split on `\n----\n`. All three behaviours were removed. Old clients that posted to `/$psql` without `execute=true` and parsed `[{:result …}, …]` need to be updated to the shape above. The endpoint URL is unchanged. + +## $psql-cancel + +Cancel an in-flight query (sync or async) by its tag UUID. + +```yaml +POST /$psql-cancel + +{ "query-id": "" } +``` + +The handler runs `pg_cancel_backend(pid)` on rows in `pg_stat_activity` whose `application_name` matches `aidbox-psql:` and returns the matched backends: + +```yaml +{ "cancelled": [ true ] } +``` + +The same endpoint covers sync and async runs because both tag the session with the same prefix. + ## SQL migrations Aidbox provides `POST and GET /db/migrations` operations to enable SQL migrations, which can be used to migrate/transform data, create helper functions, views etc. diff --git a/docs/overview/aidbox-ui/README.md b/docs/overview/aidbox-ui/README.md index 39c8bc4fd..956573f26 100644 --- a/docs/overview/aidbox-ui/README.md +++ b/docs/overview/aidbox-ui/README.md @@ -27,6 +27,15 @@ An interactive HTTP client built into Aidbox. Use it to execute REST API request Run SQL queries directly against the Aidbox database. Useful for debugging, analytics, and working with [SQL on FHIR](../../modules/sql-on-fhir/README.md) ViewDefinitions. +Per-tab settings: + +* **Transaction mode** — `Autocommit` (each statement commits immediately; required for `VACUUM`, `CREATE INDEX CONCURRENTLY`) or `Transaction` (whole script wrapped in a single transaction). +* **Timeout** — per-query `statement_timeout`, default 1 minute. +* **Limit** — JDBC fetch size cap. +* **Execution** — `Foreground` (synchronous, results returned to the UI) or `Background` (fire-and-forget; the server runs the query, the UI does not retain results). + +Stop button cancels the running query via [`$psql-cancel`](../../api/rest-api/other/sql-endpoints.md#usdpsql-cancel). The `Tab` key indents inside the editor; `EXPLAIN` plans render as a monospace block. Each tab keeps its own settings, query text, and running state. Backed by the [`$psql` endpoint](../../api/rest-api/other/sql-endpoints.md#usdpsql). + ### FHIR Packages Browse, install, and manage FHIR Implementation Guides and NPM packages loaded into Aidbox via the [FHIR Artifact Registry](../../artifact-registry/artifact-registry-overview.md). Inspect individual resources within each package. diff --git a/docs/overview/release-notes.md b/docs/overview/release-notes.md index 895dad7ba..d562875b8 100644 --- a/docs/overview/release-notes.md +++ b/docs/overview/release-notes.md @@ -7,6 +7,19 @@ description: >- # Release Notes ## April 2026 _`edge`_ +* Aidbox FHIR server + + **Features** + + * Reworked SQL Console — see the new [`$psql` endpoint](../api/rest-api/other/sql-endpoints.md#usdpsql) for the full backend contract. Configurable per-tab transaction mode (transaction / autocommit), `statement_timeout`, fetch size, foreground / background execution, and `Tab` keybinding to indent. `EXPLAIN` plans render as a monospace block. + * Background SQL execution via `X-Aidbox-Sql-Async: true` — the server runs the query without retaining result rows; only metadata (status / duration / query / error) is recorded. + * Robust query cancellation via [`$psql-cancel`](../api/rest-api/other/sql-endpoints.md#usdpsql-cancel) — matches by `application_name` set from the new `X-Aidbox-Sql-Query-Id` header instead of a fragile `LIKE '%query%'` scan. Cancels both sync and async runs. + + **Breaking changes** + + * `$psql` response shape changed from `[{:result :duration :status :query}, …]` to `{:status :result [{:type :rset|:count :data …} …] :duration :query}`. The `execute=true` query parameter and the `\n----\n` multi-statement separator are no longer recognised. `$sql` is unchanged. See [SQL endpoints](../api/rest-api/other/sql-endpoints.md#usdpsql). + * The legacy DB Console at `/ui/db` now redirects to the new SQL Console at `/u/db-console` and forwards the `?query=` URL parameter. + ## March 2026 _`latest, 2603`_ * Aidbox FHIR server From be39a1444f52d2e97ffadf88337174b2bf74aedd Mon Sep 17 00:00:00 2001 From: Andrey Listopadov Date: Wed, 29 Apr 2026 11:33:49 +0300 Subject: [PATCH 2/6] Drop X- prefix from SQL header names --- docs/api/rest-api/other/sql-endpoints.md | 12 ++++++------ docs/overview/release-notes.md | 4 ++-- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/api/rest-api/other/sql-endpoints.md b/docs/api/rest-api/other/sql-endpoints.md index e16e8b6ec..a5ef631bf 100644 --- a/docs/api/rest-api/other/sql-endpoints.md +++ b/docs/api/rest-api/other/sql-endpoints.md @@ -84,11 +84,11 @@ Every header below is optional. Defaults match a single-transaction read-write r | Header | Value | Effect | |---|---|---| -| `X-Aidbox-Sql-Autocommit` | `true` | Run outside a transaction. Required for `VACUUM`, `CREATE INDEX CONCURRENTLY`, `REINDEX CONCURRENTLY`. Rejected when [`db.pass-auth-vars`](../../../reference/all-settings.md#db.pass-auth-vars) is on **and** the request carries a resolvable identity — autocommit would drop the SQL identity injection that RLS relies on. | -| `X-Aidbox-Sql-Timeout` | seconds, 1..86400 | Per-query `statement_timeout`. Empty / non-numeric / negative / out-of-range values are ignored. | -| `X-Aidbox-Sql-Read-Only` | `true` | Run as read-only. Writes raise `ERROR: cannot execute … in a read-only transaction`. | -| `X-Aidbox-Sql-Query-Id` | UUID | Tags the PG session via `application_name = aidbox-psql:`. The same UUID is used to cancel via `/$psql-cancel`. | -| `X-Aidbox-Sql-Async` | `true` | Fire-and-forget background execution. Returns `202 { "operation-id": "" }` immediately. The query runs server-side; result rows are not retained — only `status` / `duration` / `query` / `error` are kept in `db_scheduler.scheduled_tasks_history`. The same handler accepts the operation-id as a `query-id` for cancellation. | +| `Aidbox-Sql-Autocommit` | `true` | Run outside a transaction. Required for `VACUUM`, `CREATE INDEX CONCURRENTLY`, `REINDEX CONCURRENTLY`. Rejected when [`db.pass-auth-vars`](../../../reference/all-settings.md#db.pass-auth-vars) is on **and** the request carries a resolvable identity — autocommit would drop the SQL identity injection that RLS relies on. | +| `Aidbox-Sql-Timeout` | seconds, 1..86400 | Per-query `statement_timeout`. Empty / non-numeric / negative / out-of-range values are ignored. | +| `Aidbox-Sql-Read-Only` | `true` | Run as read-only. Writes raise `ERROR: cannot execute … in a read-only transaction`. | +| `Aidbox-Sql-Query-Id` | UUID | Tags the PG session via `application_name = aidbox-psql:`. The same UUID is used to cancel via `/$psql-cancel`. | +| `Aidbox-Sql-Async` | `true` | Fire-and-forget background execution. Returns `202 { "operation-id": "" }` immediately. The query runs server-side; result rows are not retained — only `status` / `duration` / `query` / `error` are kept in `db_scheduler.scheduled_tasks_history`. The same handler accepts the operation-id as a `query-id` for cancellation. | ### Breaking change in 2604 @@ -101,7 +101,7 @@ Cancel an in-flight query (sync or async) by its tag UUID. ```yaml POST /$psql-cancel -{ "query-id": "" } +{ "query-id": "" } ``` The handler runs `pg_cancel_backend(pid)` on rows in `pg_stat_activity` whose `application_name` matches `aidbox-psql:` and returns the matched backends: diff --git a/docs/overview/release-notes.md b/docs/overview/release-notes.md index d562875b8..90d629e24 100644 --- a/docs/overview/release-notes.md +++ b/docs/overview/release-notes.md @@ -12,8 +12,8 @@ description: >- **Features** * Reworked SQL Console — see the new [`$psql` endpoint](../api/rest-api/other/sql-endpoints.md#usdpsql) for the full backend contract. Configurable per-tab transaction mode (transaction / autocommit), `statement_timeout`, fetch size, foreground / background execution, and `Tab` keybinding to indent. `EXPLAIN` plans render as a monospace block. - * Background SQL execution via `X-Aidbox-Sql-Async: true` — the server runs the query without retaining result rows; only metadata (status / duration / query / error) is recorded. - * Robust query cancellation via [`$psql-cancel`](../api/rest-api/other/sql-endpoints.md#usdpsql-cancel) — matches by `application_name` set from the new `X-Aidbox-Sql-Query-Id` header instead of a fragile `LIKE '%query%'` scan. Cancels both sync and async runs. + * Background SQL execution via `Aidbox-Sql-Async: true` — the server runs the query without retaining result rows; only metadata (status / duration / query / error) is recorded. + * Robust query cancellation via [`$psql-cancel`](../api/rest-api/other/sql-endpoints.md#usdpsql-cancel) — matches by `application_name` set from the new `Aidbox-Sql-Query-Id` header instead of a fragile `LIKE '%query%'` scan. Cancels both sync and async runs. **Breaking changes** From d227ddeb65401298f94a6cad280e9d56b6081071 Mon Sep 17 00:00:00 2001 From: Andrey Listopadov Date: Wed, 29 Apr 2026 11:42:06 +0300 Subject: [PATCH 3/6] Address review comments on $psql docs --- docs/api/rest-api/other/sql-endpoints.md | 10 ++++++---- docs/overview/release-notes.md | 10 +++++----- 2 files changed, 11 insertions(+), 9 deletions(-) diff --git a/docs/api/rest-api/other/sql-endpoints.md b/docs/api/rest-api/other/sql-endpoints.md index a5ef631bf..5a0c026b5 100644 --- a/docs/api/rest-api/other/sql-endpoints.md +++ b/docs/api/rest-api/other/sql-endpoints.md @@ -41,7 +41,7 @@ POST /$sql?_format=yaml ## $psql -Run a raw multi-statement SQL script in a single request. The endpoint is what the Aidbox UI SQL Console uses; `/$notebook-psql` is an alias with identical behavior. +Run a raw multi-statement SQL script in a single request. The Aidbox UI SQL Console uses this endpoint. Request body: @@ -51,8 +51,8 @@ POST /$psql { "query": "SELECT 1; SELECT 2", "limit": 1000 } ``` -- `query` — the SQL text. Sent verbatim to PostgreSQL; Aidbox does not split, trim, or rewrite it. -- `limit` (optional) — applied via JDBC `setMaxRows` to cap each result set. +- `query` — the SQL text. Aidbox sends the query verbatim, without splitting, trimming, or rewriting it. +- `limit` (optional) — caps the number of rows returned per statement. Response (success): @@ -92,7 +92,9 @@ Every header below is optional. Defaults match a single-transaction read-write r ### Breaking change in 2604 -Prior versions returned a vector of per-statement debug objects and accepted an `execute=true` query parameter to switch between two execution paths; multi-statement scripts were split on `\n----\n`. All three behaviours were removed. Old clients that posted to `/$psql` without `execute=true` and parsed `[{:result …}, …]` need to be updated to the shape above. The endpoint URL is unchanged. +`$psql` only — `$sql` is unaffected. + +Prior versions of `$psql` returned a vector of per-statement debug objects and accepted an `execute=true` query parameter that switched between two execution paths; multi-statement scripts were split on `\n----\n`. Aidbox removed all three behaviours. Old clients that posted to `/$psql` without `execute=true` and parsed `[{:result …}, …]` need to update to the response shape above. The endpoint URL is unchanged. ## $psql-cancel diff --git a/docs/overview/release-notes.md b/docs/overview/release-notes.md index 90d629e24..66b3900f1 100644 --- a/docs/overview/release-notes.md +++ b/docs/overview/release-notes.md @@ -11,14 +11,14 @@ description: >- **Features** - * Reworked SQL Console — see the new [`$psql` endpoint](../api/rest-api/other/sql-endpoints.md#usdpsql) for the full backend contract. Configurable per-tab transaction mode (transaction / autocommit), `statement_timeout`, fetch size, foreground / background execution, and `Tab` keybinding to indent. `EXPLAIN` plans render as a monospace block. - * Background SQL execution via `Aidbox-Sql-Async: true` — the server runs the query without retaining result rows; only metadata (status / duration / query / error) is recorded. - * Robust query cancellation via [`$psql-cancel`](../api/rest-api/other/sql-endpoints.md#usdpsql-cancel) — matches by `application_name` set from the new `Aidbox-Sql-Query-Id` header instead of a fragile `LIKE '%query%'` scan. Cancels both sync and async runs. + * Reworked SQL Console — per-tab transaction mode (transaction / autocommit), `statement_timeout`, foreground / background execution, and a `Tab` keybinding that indents. + * Background SQL execution via `Aidbox-Sql-Async: true`. The server runs the query without retaining result rows. + * Query cancellation via [`$psql-cancel`](../api/rest-api/other/sql-endpoints.md#usdpsql-cancel). **Breaking changes** - * `$psql` response shape changed from `[{:result :duration :status :query}, …]` to `{:status :result [{:type :rset|:count :data …} …] :duration :query}`. The `execute=true` query parameter and the `\n----\n` multi-statement separator are no longer recognised. `$sql` is unchanged. See [SQL endpoints](../api/rest-api/other/sql-endpoints.md#usdpsql). - * The legacy DB Console at `/ui/db` now redirects to the new SQL Console at `/u/db-console` and forwards the `?query=` URL parameter. + * [`$psql` response shape changed](../api/rest-api/other/sql-endpoints.md#breaking-change-in-2604). The `execute=true` query parameter and the `\n----\n` multi-statement separator are no longer recognised. `$sql` is unchanged. + * The legacy DB Console at `/ui/db` now redirects to the new SQL Console at `/u/db-console`. ## March 2026 _`latest, 2603`_ From 99259642ab9f4206e87fda28a32ccbccf2d1cf3e Mon Sep 17 00:00:00 2001 From: Andrey Listopadov Date: Wed, 29 Apr 2026 12:43:29 +0300 Subject: [PATCH 4/6] [#7417] Document $psql access scope and persistence trail MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The endpoint persists SQL text and identifying request metadata into multiple stores (logs, audit, scheduler history, pg_stat_activity). Make explicit that $psql is privileged-by-design — exposing it to non-admin roles broadens access to logs and audit, not just to data. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/api/rest-api/other/sql-endpoints.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/api/rest-api/other/sql-endpoints.md b/docs/api/rest-api/other/sql-endpoints.md index 5a0c026b5..d7041a65f 100644 --- a/docs/api/rest-api/other/sql-endpoints.md +++ b/docs/api/rest-api/other/sql-endpoints.md @@ -90,6 +90,17 @@ Every header below is optional. Defaults match a single-transaction read-write r | `Aidbox-Sql-Query-Id` | UUID | Tags the PG session via `application_name = aidbox-psql:`. The same UUID is used to cancel via `/$psql-cancel`. | | `Aidbox-Sql-Async` | `true` | Fire-and-forget background execution. Returns `202 { "operation-id": "" }` immediately. The query runs server-side; result rows are not retained — only `status` / `duration` / `query` / `error` are kept in `db_scheduler.scheduled_tasks_history`. The same handler accepts the operation-id as a `query-id` for cancellation. | +### Access scope + +`$psql` is privileged-by-design. The SQL text a caller sends lands in several stores beyond the immediate response: + +- `pg_stat_activity` while the query is in flight (visible to anyone with `pg_read_all_stats` or superuser). +- `klog` `:db/q` events — every successful run is logged, truncated by `proto.klog/sql-max-length` (default 500 chars). +- `AuditEvent` resources via the standard audit pipeline (base64-encoded SQL text). +- `db_scheduler.scheduled_tasks_history` rows for `Aidbox-Sql-Async: true` runs (until the row is cleaned up). + +Anyone with permission to call `$psql` therefore has effective access to whatever the SQL itself reads, plus a long-lived trail in logs, audit, and scheduler history. Treat the endpoint as superuser-equivalent — do not expose it to non-admin roles via AccessPolicy. + ### Breaking change in 2604 `$psql` only — `$sql` is unaffected. From 5ee9a5f86828eedfab560ba40dc23187c03ef131 Mon Sep 17 00:00:00 2001 From: Andrey Listopadov Date: Wed, 29 Apr 2026 14:41:54 +0300 Subject: [PATCH 5/6] remove line --- docs/api/rest-api/other/sql-endpoints.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/docs/api/rest-api/other/sql-endpoints.md b/docs/api/rest-api/other/sql-endpoints.md index d7041a65f..e3370b55e 100644 --- a/docs/api/rest-api/other/sql-endpoints.md +++ b/docs/api/rest-api/other/sql-endpoints.md @@ -8,7 +8,7 @@ description: Execute raw SQL queries directly in Aidbox via $sql, $psql, and $ps Execute SQL in Aidbox. Supported params: -- SQL string +- SQL string - jdbc friendly array [SQL, param, param] Example request: @@ -20,7 +20,7 @@ POST /$sql?_format=yaml SELECT count(*) FROM patient -# Response +# Response # # - {count: 7} ``` @@ -32,7 +32,7 @@ POST /$sql?_format=yaml ["SELECT count(*) FROM patient where resource->'status' = ?", true] -# Response +# Response # # - {count: 2} ``` @@ -103,8 +103,6 @@ Anyone with permission to call `$psql` therefore has effective access to whateve ### Breaking change in 2604 -`$psql` only — `$sql` is unaffected. - Prior versions of `$psql` returned a vector of per-statement debug objects and accepted an `execute=true` query parameter that switched between two execution paths; multi-statement scripts were split on `\n----\n`. Aidbox removed all three behaviours. Old clients that posted to `/$psql` without `execute=true` and parsed `[{:result …}, …]` need to update to the response shape above. The endpoint URL is unchanged. ## $psql-cancel @@ -139,7 +137,7 @@ POST /db/migrations update patient set resource = resource - 'extension' - id: create-policy-helper sql: | - create function patient_for_user(u jsonb) returns jsonb + create function patient_for_user(u jsonb) returns jsonb as $$ select resource || jsonb_build_object('id', id) from patient @@ -151,7 +149,7 @@ POST /db/migrations sql: ... - id: create-policy-helper sql: ... - + -- second run response [] ``` From ab15ba5c60104020db49f3c6683d77a0f521576a Mon Sep 17 00:00:00 2001 From: Andrey Listopadov Date: Wed, 29 Apr 2026 14:42:48 +0300 Subject: [PATCH 6/6] [#7417] Link Aidbox-Sql-Async release note to $psql docs Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/overview/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/overview/release-notes.md b/docs/overview/release-notes.md index 66b3900f1..1dfb2bac8 100644 --- a/docs/overview/release-notes.md +++ b/docs/overview/release-notes.md @@ -12,7 +12,7 @@ description: >- **Features** * Reworked SQL Console — per-tab transaction mode (transaction / autocommit), `statement_timeout`, foreground / background execution, and a `Tab` keybinding that indents. - * Background SQL execution via `Aidbox-Sql-Async: true`. The server runs the query without retaining result rows. + * Background SQL execution via [`Aidbox-Sql-Async: true`](../api/rest-api/other/sql-endpoints.md#usdpsql). The server runs the query without retaining result rows. * Query cancellation via [`$psql-cancel`](../api/rest-api/other/sql-endpoints.md#usdpsql-cancel). **Breaking changes**