Skip to content

Commit d743473

Browse files
authored
fix(clickhouse): drop obsolete task event index variants (#4880)
## Summary Removes the obsolete task event full-text indexes in one `ALTER TABLE`, accounting for both known index naming variants. ## Fix The original migration issued back-to-back alters while ClickHouse could still be processing the preceding index mutation. Combining the index removals avoids scheduling a subsequent alter against an unfinished mutation. The native JSON storage change is deferred until it can be performed without rewriting populated parts.
1 parent cc0049c commit d743473

2 files changed

Lines changed: 14 additions & 19 deletions

File tree

internal-packages/clickhouse/schema/042_reduce_task_events_v2_storage_overhead.sql

Lines changed: 4 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -3,19 +3,9 @@
33
-- Full-text search is served outside the source event table. Keeping these
44
-- indexes here adds work to every event insert and merge without serving reads.
55
ALTER TABLE trigger_dev.task_events_v2
6-
DROP INDEX IF EXISTS idx_attributes_text_search;
7-
8-
ALTER TABLE trigger_dev.task_events_v2
9-
DROP INDEX IF EXISTS idx_message_text_search;
10-
11-
-- attributes remains an insert input for attributes_text, but is no longer
12-
-- stored. Writers must include attributes in an explicit insert column list
13-
-- because implicit INSERT column lists exclude EPHEMERAL columns.
14-
ALTER TABLE trigger_dev.task_events_v2
15-
MODIFY COLUMN attributes JSON EPHEMERAL;
6+
DROP INDEX IF EXISTS idx_attributes_text_search,
7+
DROP INDEX IF EXISTS idx_attributes_text,
8+
DROP INDEX IF EXISTS idx_message_text_search,
9+
DROP INDEX IF EXISTS message_text_search;
1610

1711
-- +goose Down
18-
19-
-- Restoring the stored JSON column safely requires inspecting the live schema
20-
-- and coordinating the writer rollback. Use a new forward migration instead.
21-
SELECT throwIf(1, 'This migration cannot be rolled back automatically');

internal-packages/clickhouse/src/taskEvents.test.ts

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ describe("task events v2", () => {
1313
const ch = new ClickHouse({ url: clickhouseContainer.getConnectionUrl(), name: "test" });
1414
const startTime = new Date("2026-09-01T10:00:00.000Z");
1515
const expiresAt = new Date(Date.now() + 90 * 24 * 60 * 60 * 1000);
16-
const spanId = "span_ephemeral_attributes";
16+
const spanId = "span_materialized_attributes";
1717

1818
const [insertError] = await ch.taskEventsV2.insert([
1919
{
@@ -42,7 +42,7 @@ describe("task events v2", () => {
4242
expect(insertError).toBeNull();
4343

4444
const readAttributes = ch.reader.query({
45-
name: "read-ephemeral-task-event-attributes",
45+
name: "read-task-event-attributes",
4646
query: `SELECT attributes_text,
4747
toUInt8(inserted_at > toDateTime64('2020-01-01 00:00:00', 3)) AS has_inserted_at
4848
FROM trigger_dev.task_events_v2
@@ -82,8 +82,8 @@ describe("task events v2", () => {
8282
expect(columns).toEqual([
8383
{
8484
name: "attributes",
85-
default_kind: "EPHEMERAL",
86-
default_expression: "defaultValueOfTypeName('JSON')",
85+
default_kind: "",
86+
default_expression: "",
8787
},
8888
{
8989
name: "attributes_text",
@@ -98,7 +98,12 @@ describe("task events v2", () => {
9898
FROM system.data_skipping_indices
9999
WHERE database = 'trigger_dev'
100100
AND table = 'task_events_v2'
101-
AND name IN ('idx_attributes_text_search', 'idx_message_text_search')
101+
AND name IN (
102+
'idx_attributes_text_search',
103+
'idx_attributes_text',
104+
'idx_message_text_search',
105+
'message_text_search'
106+
)
102107
ORDER BY name`,
103108
schema: z.object({ name: z.string() }),
104109
});

0 commit comments

Comments
 (0)