Skip to content

Commit 790d93a

Browse files
authored
feat(tables): live collaboration — cell-selection presence + live mutation propagation (#5957)
* feat(tables): live cell-selection presence — protocol + server + client hook The realtime spine for Google-Sheets-style table presence (mode A, socket): - @sim/realtime-protocol/table-presence: centralized wire protocol (events + TableCellSelection {anchor, focus, editing} + payloads) so server emits and client subscriptions can't drift. - ROOM_TYPES.TABLE + resolveTableWorkspace registered in ROOM_WORKSPACE_RESOLVERS (tableId -> workspace via userTableDefinitions, honoring archivedAt); roomName / presenceEventName / disconnect cleanup / authorizeRoom all derive automatically. - apps/realtime/src/handlers/tables.ts: join/leave (mirrors workspace-files) + a table-cell-selection relay (mirrors the workflow selection channel), broadcasting via roomName(room) since table rooms are namespaced. UserPresence gains a cell field threaded through the memory + Redis managers (Lua ARGV[7], null clears). - Extracted the duplicated resolveAvatarUrl into handlers/avatar.ts. - use-table-room.ts client hook: joins over the shared socket, tracks the roster (avatars) + patches per-socket cell deltas, exposes a throttled emitCellSelection. Grid UI (avatars + selection overlay) lands next; concurrent cell-value edits (last-write-wins via the durable log) are the follow-up PR. * feat(tables): render live cell-selection presence in the grid Wires the table presence room into the grid UI: - Page (table.tsx): useTableRoom (gated off in embedded/mothership mode) — renders <PresenceAvatars> in the header and passes remoteSelections + emitCellSelection down to the grid. - Grid emits its local selection: an effect resolves the index-based anchor/focus to stable (rowId, columnId) via refs and broadcasts it (with an editing flag for the active cell) through the throttled emitter. - RemoteSelectionOverlay: draws each remote viewer's selection in their color (getUserColor), a darker fill while editing, and name-on-hover — measured from live cell rects in the content wrapper's space (scrolls with the grid), hidden when rows are virtualized off-window, pointer-events-none so it never blocks cell clicks (hover via pointer hit-test). * test(tables): cover the table presence handler Mirrors workspace-files.test.ts: join auth/unavailable/denied/success, plus the cell-selection relay (asserts it persists via updateUserActivity and broadcasts on the namespaced roomName, not the bare id) and leave. * feat(tables): propagate manual cell edits live (last-write-wins) A manual row edit now appends a lightweight 'edit' event to the durable table stream; collaborators refetch the row (via the existing debounced rows-invalidate the job events use) so the winning value shows live. The event carries no value — peers refetch in their own wire format, so there's no auth-specific value translation on the wire, and last-write-wins falls out of the DB's committed order (the Google-Sheets model). Edits that also trigger a dispatch already emit dispatch/cell events; the debounce coalesces the two. * refactor(tables): apply /simplify findings - Drop the dead 'add unknown peer' upsert branch in use-table-room (Socket.IO ordering guarantees a peer is in the roster before their selection delta). - TableCellSelectionBroadcast = TablePresenceUser & { cell } (was a copy-paste). - Make TableGrid's presence props required + drop the unused empty-default/guard (only table.tsx mounts it, always passing both). - Drop the unused rowId from the 'edit' event (the handler invalidates all rows). - Overlay: subscribe scroll/resize/pointer listeners once per scroll element and cache the wrapper origin, so incoming deltas re-measure without re-subscribing and the pointer hit-test never forces a per-move layout read. - Server: cache the immutable socket session so a selection delta no longer reads it from Redis every time. * refactor(tables): apply /cleanup findings - Fix the remote-selection name label contrast: text-white is unreadable on the light-pastel user colors (same bug the Files caret fixed) → fixed dark #1a1a1a. - Re-measure via useLayoutEffect so a moving peer selection updates before paint (no one-frame position lag). - Drop 'mothership' from a comment (constitution copy rule). Six cleanup passes ran (effect, memo/callback, state, react-query, emcn, comment); the rest confirmed clean — all state/memos/callbacks/effects are load-bearing, presence correctly lives in useState (socket-pushed), and the edit→rows-invalidate granularity is right. * feat(tables): propagate every table mutation live (edit + schema signals) Comprehensive live collaboration for all user table mutations, via two value-less durable signals + named helpers (signalTableRowsChanged / signalTableSchemaChanged): - edit (rows refetch): single + batch row create, cell/row update, batch update, delete by id/filter, and upsert. - schema (definition + rows refetch): column add/update/delete, workflow-group add/update/delete, table rename, and CSV import (which can add columns). - Client handles 'schema' by invalidating the table detail (exact) + rows. Execution paths (column run, cancel-runs) and async jobs (delete/import-async, job-cancel) already propagate via cell/dispatch/job events — verified applyJob refetches on terminal. No reorder routes exist. Table archive (route DELETE) is a deliberate follow-up: it needs a table-deleted redirect event, not a refetch signal (which would 404). * refactor(tables): apply comprehensive /cleanup audit findings Holistic + react-query + comment audits over the whole PR: - Security/crash fix: a remote peer's rowId flowed unescaped into the overlay's querySelector — a hostile id ('x"]') threw SyntaxError inside a useLayoutEffect, crashing every other viewer's page. CSS.escape it, and validate + whitelist the untrusted cell payload server-side (shape + 200-char id bound) before it is stored/rebroadcast. - Simplify the CELL_SELECTION relay: the delta attached userId/userName/avatarUrl that the client discarded (identity comes from the roster). Drop them + the getUserSession lookup/cache entirely — the delta is now { socketId, cell }. - React Query: schema handler also invalidates lists() (parity with the local column-mutation set); document that the mutating client self-refetches by design. - Comment tightenings; biome fixed a stale import order in workspace-files.ts. * fix(tables): broadcast single-cell selections (focus falls back to anchor) Cursor High: a normal cell click leaves selectionFocus null (the grid treats it as a one-cell selection via focus ?? anchor), but the presence emit required BOTH anchor and focus to resolve — so the most common selection never broadcast and clicking even cleared a prior remote outline. Mirror the grid's focus ?? anchor semantics. * fix(tables): reviewer + regression + per-LOC audit findings Cursor review round (5 findings) + regression audit + per-LOC audit: - Presence roster snapshot now KEEPS the cell we already hold for a known socket, so a join/leave broadcast can't revert a fresher CELL_SELECTION delta. - Reset the selection throttle on table switch (was unmount-only), so a pending selection for table A can't flush into table B's room after a switch. - Metadata writes (column widths, display) use a new lightweight 'metadata' signal that refetches only the definition — a resize no longer forces peers to refetch rows. - Overlay re-measures on row add/remove/reorder via a tbody childList MutationObserver (a live refetch moves cells without a scroll/resize). - Document the actor self-refetch create caveat (scrolled multi-page insert) accurately. - isCellRef narrows to a partial instead of casting to the full type then re-checking; drop a redundant mount measure() (the layout effect covers it); text-[11px]→text-xs. * fix(tables): drop ineffective metadata propagation + re-measure overlay on column resize Cursor round on b8f28b0: - Remove the 'metadata' signal entirely. The grid seeds columnWidths/pinnedColumns from metadata ONCE (metadataSeededRef) and deliberately never re-applies them (to avoid clobbering a local in-progress resize), so refetching the definition on a peer never surfaced their width/pin change — an ineffective path. Width/pin live-sync needs reconciliation that doesn't clobber a local resize; that's a deliberate follow-up, not a no-op refetch. Structural changes still propagate via 'schema'. - Overlay now also observes the content layer with the ResizeObserver, so a column resize (which grows the content, not the scroll container) re-measures remote outlines. - Presence-merge comment now states both sides of the trade-off. * fix(tables): re-broadcast local selection on (re)join Cursor Medium: a selection made before the room join completes (or held across a reconnect) was dropped server-side and never re-sent, so peers didn't see it until the local user moved it again. Track the current selection in a ref (set on every emit, cleared on table switch) and re-emit it from handleJoinSuccess once the room is joined. * fix(tables): re-broadcast selection when a peer's row change shifts it End-to-end lifecycle audit (Low-Med): the selection emit resolved the stable (rowId, columnId) only on selection/editing change, not when a live edit/schema refetch inserted/deleted/reordered rows. The index-based local selection then sat on a different logical row than the rowId peers held, so your outline showed on the old row until you moved. Re-run the emit on rows/displayColumns change and dedup an unchanged result (also drops the redundant null-on-open emit) so the broadcast stays consistent with the local highlight. * fix(tables): schema invalidates run-state/enrichment + guard stale join Cursor round on cdc8796 (2 Medium): - schema handler used detail exact:true, so it skipped the activeDispatches + enrichmentDetails sibling queries the local invalidateTableSchema refreshes via a prefix match. After a peer deletes/restructures a workflow group, peers could keep a stale running badge or enrichment panel. Now invalidates both siblings too (rows stay on the debounce). - Guard against a stale join stealing the room: a fast table A->B switch could let A's async authorize finish after B, leave B, and strand the socket in A. Added a per-socket monotonic join generation checked after authorize (mirrors the file-doc relay's guard) + a test. * feat(tables): live column width/pin/order sync Collaborators now see each other's column resizes, pins, and reorders live — the last piece of Google-Sheets-style layout parity. - New lightweight `metadata` durable event kind (distinct from `schema`): only the table definition carries UI metadata, so peers refetch the definition alone — no rows/run-state refetch. The metadata PUT route now signals it. - The grid reconciles server metadata against its in-progress gesture: the column being actively resized keeps its live local width, and an in-flight column drag blocks a reorder apply — so a peer's change never reverts the local action. Each field is reference-guarded (React Query structural sharing keeps unchanged sub-objects stable), so an unrelated peer change doesn't re-apply the others. * fix(tables): escalate to schema signal when a reorder scrubs group deps Independent audit of the metadata-sync commit found a stale-run-state hole: a columnOrder PUT that moves a column left of a workflow group's leftmost column makes updateTableMetadata scrub that group's dependencies and write a new schema — a real structural change. But the route only fired the lightweight 'metadata' signal (detail-only refetch), so peers' and the actor's activeDispatches / enrichmentDetails queries stayed stale (a lingering running badge / enrichment panel) — exactly what the 'schema' handler exists to prevent. updateTableMetadata now reports whether it scrubbed the schema; the route emits signalTableSchemaChanged in that case and the light signalTableMetadataChanged otherwise. Width/pin/plain-reorder stay on the cheap detail-only path.
1 parent 45cb10b commit 790d93a

28 files changed

Lines changed: 1292 additions & 43 deletions

File tree

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
import { db, user } from '@sim/db'
2+
import { createLogger } from '@sim/logger'
3+
import { eq } from 'drizzle-orm'
4+
import type { AuthenticatedSocket } from '@/middleware/auth'
5+
6+
const logger = createLogger('PresenceAvatar')
7+
8+
/**
9+
* The avatar URL for a presence entry: the socket's authenticated image when
10+
* present, otherwise a single lookup of the user's stored image. Never throws —
11+
* presence must not fail on an avatar lookup, so a DB error resolves to `null`.
12+
*/
13+
export async function resolveAvatarUrl(
14+
socket: AuthenticatedSocket,
15+
userId: string
16+
): Promise<string | null> {
17+
if (socket.userImage) return socket.userImage
18+
try {
19+
const [record] = await db
20+
.select({ image: user.image })
21+
.from(user)
22+
.where(eq(user.id, userId))
23+
.limit(1)
24+
return record?.image ?? null
25+
} catch (error) {
26+
logger.warn('Failed to load user avatar for presence', { userId, error })
27+
return null
28+
}
29+
}

apps/realtime/src/handlers/index.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ import { setupWorkspaceFileDocHandlers } from '@/handlers/file-doc'
33
import { setupOperationsHandlers } from '@/handlers/operations'
44
import { setupPresenceHandlers } from '@/handlers/presence'
55
import { setupSubblocksHandlers } from '@/handlers/subblocks'
6+
import { setupTablesHandlers } from '@/handlers/tables'
67
import { setupVariablesHandlers } from '@/handlers/variables'
78
import { setupWorkflowHandlers } from '@/handlers/workflow'
89
import { setupWorkspaceFilesHandlers } from '@/handlers/workspace-files'
@@ -17,5 +18,6 @@ export function setupAllHandlers(socket: AuthenticatedSocket, roomManager: IRoom
1718
setupPresenceHandlers(socket, roomManager)
1819
setupWorkspaceFilesHandlers(socket, roomManager)
1920
setupWorkspaceFileDocHandlers(socket, roomManager)
21+
setupTablesHandlers(socket, roomManager)
2022
setupConnectionHandlers(socket, roomManager)
2123
}
Lines changed: 234 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,234 @@
1+
/**
2+
* @vitest-environment node
3+
*/
4+
import { ROOM_TYPES } from '@sim/realtime-protocol/rooms'
5+
import { TABLE_PRESENCE_EVENTS } from '@sim/realtime-protocol/table-presence'
6+
import { beforeEach, describe, expect, it, vi } from 'vitest'
7+
import type { IRoomManager } from '@/rooms'
8+
9+
const { mockAuthorizeRoom } = vi.hoisted(() => ({
10+
mockAuthorizeRoom: vi.fn(),
11+
}))
12+
13+
vi.mock('@sim/db', () => ({
14+
db: { select: vi.fn() },
15+
user: { image: 'image' },
16+
}))
17+
18+
vi.mock('@sim/platform-authz/rooms', () => ({
19+
authorizeRoom: mockAuthorizeRoom,
20+
}))
21+
22+
import { setupTablesHandlers } from '@/handlers/tables'
23+
24+
const TABLE_ROOM = { type: ROOM_TYPES.TABLE, id: 'table-1' }
25+
26+
function createSocket(overrides?: Record<string, unknown>) {
27+
const handlers: Record<string, (payload: unknown) => Promise<void> | void> = {}
28+
const toEmit = vi.fn()
29+
const socket = {
30+
id: 'socket-1',
31+
userId: 'user-1',
32+
userName: 'Test User',
33+
userImage: 'avatar.png',
34+
on: vi.fn((event: string, handler: (payload: unknown) => Promise<void> | void) => {
35+
handlers[event] = handler
36+
}),
37+
emit: vi.fn(),
38+
join: vi.fn(),
39+
leave: vi.fn(),
40+
to: vi.fn().mockReturnValue({ emit: toEmit }),
41+
...overrides,
42+
}
43+
return { handlers, socket, toEmit }
44+
}
45+
46+
function createRoomManager(overrides?: Partial<IRoomManager>): IRoomManager {
47+
return {
48+
isReady: vi.fn().mockReturnValue(true),
49+
getRoomForSocket: vi.fn().mockResolvedValue(null),
50+
getRoomsForSocket: vi.fn().mockResolvedValue([]),
51+
removeUserFromRoom: vi.fn().mockResolvedValue(false),
52+
removeSocketFromAllRooms: vi.fn().mockResolvedValue([]),
53+
broadcastPresenceUpdate: vi.fn().mockResolvedValue(undefined),
54+
getRoomUsers: vi.fn().mockResolvedValue([]),
55+
hasRoom: vi.fn().mockResolvedValue(false),
56+
deleteRoom: vi.fn().mockResolvedValue(undefined),
57+
addUserToRoom: vi.fn().mockResolvedValue(undefined),
58+
getUserSession: vi.fn().mockResolvedValue(null),
59+
updateUserActivity: vi.fn().mockResolvedValue(undefined),
60+
updateRoomLastModified: vi.fn().mockResolvedValue(undefined),
61+
emitToRoom: vi.fn(),
62+
getUniqueUserCount: vi.fn().mockResolvedValue(1),
63+
getTotalActiveConnections: vi.fn().mockResolvedValue(0),
64+
shutdown: vi.fn().mockResolvedValue(undefined),
65+
initialize: vi.fn().mockResolvedValue(undefined),
66+
io: {
67+
in: vi.fn().mockReturnValue({ socketsLeave: vi.fn().mockResolvedValue(undefined) }),
68+
},
69+
...overrides,
70+
} as unknown as IRoomManager
71+
}
72+
73+
type SetupArg = Parameters<typeof setupTablesHandlers>[0]
74+
75+
describe('setupTablesHandlers', () => {
76+
beforeEach(() => {
77+
vi.clearAllMocks()
78+
mockAuthorizeRoom.mockResolvedValue({
79+
allowed: true,
80+
status: 200,
81+
workspaceId: 'ws-1',
82+
workspacePermission: 'admin',
83+
})
84+
})
85+
86+
it('rejects join when the socket is not authenticated', async () => {
87+
const { socket, handlers } = createSocket({ userId: undefined, userName: undefined })
88+
setupTablesHandlers(socket as unknown as SetupArg, createRoomManager())
89+
90+
await handlers[TABLE_PRESENCE_EVENTS.JOIN]({ tableId: 'table-1' })
91+
92+
expect(socket.emit).toHaveBeenCalledWith(TABLE_PRESENCE_EVENTS.JOIN_ERROR, {
93+
tableId: 'table-1',
94+
error: 'Authentication required',
95+
code: 'AUTHENTICATION_REQUIRED',
96+
retryable: false,
97+
})
98+
})
99+
100+
it('rejects join with a retryable error when realtime is unavailable', async () => {
101+
const { socket, handlers } = createSocket()
102+
setupTablesHandlers(
103+
socket as unknown as SetupArg,
104+
createRoomManager({ isReady: vi.fn().mockReturnValue(false) })
105+
)
106+
107+
await handlers[TABLE_PRESENCE_EVENTS.JOIN]({ tableId: 'table-1' })
108+
109+
expect(socket.emit).toHaveBeenCalledWith(
110+
TABLE_PRESENCE_EVENTS.JOIN_ERROR,
111+
expect.objectContaining({ code: 'ROOM_MANAGER_UNAVAILABLE', retryable: true })
112+
)
113+
})
114+
115+
it('rejects join when table access is denied', async () => {
116+
mockAuthorizeRoom.mockResolvedValue({
117+
allowed: false,
118+
status: 403,
119+
workspaceId: 'ws-1',
120+
workspacePermission: null,
121+
})
122+
const { socket, handlers } = createSocket()
123+
setupTablesHandlers(socket as unknown as SetupArg, createRoomManager())
124+
125+
await handlers[TABLE_PRESENCE_EVENTS.JOIN]({ tableId: 'table-1' })
126+
127+
expect(socket.emit).toHaveBeenCalledWith(
128+
TABLE_PRESENCE_EVENTS.JOIN_ERROR,
129+
expect.objectContaining({ code: 'ACCESS_DENIED', retryable: false })
130+
)
131+
})
132+
133+
it('joins the table room and broadcasts presence on success', async () => {
134+
const { socket, handlers } = createSocket()
135+
const roomManager = createRoomManager()
136+
setupTablesHandlers(socket as unknown as SetupArg, roomManager)
137+
138+
await handlers[TABLE_PRESENCE_EVENTS.JOIN]({ tableId: 'table-1', tabSessionId: 'tab-1' })
139+
140+
expect(socket.join).toHaveBeenCalledWith('table:table-1')
141+
expect(roomManager.addUserToRoom).toHaveBeenCalledWith(
142+
TABLE_ROOM,
143+
'socket-1',
144+
expect.objectContaining({ userId: 'user-1', role: 'admin' })
145+
)
146+
expect(socket.emit).toHaveBeenCalledWith(
147+
TABLE_PRESENCE_EVENTS.JOIN_SUCCESS,
148+
expect.objectContaining({ tableId: 'table-1', socketId: 'socket-1' })
149+
)
150+
expect(roomManager.broadcastPresenceUpdate).toHaveBeenCalledWith(TABLE_ROOM)
151+
})
152+
153+
it('persists and relays a cell selection to the namespaced room (id + cell only)', async () => {
154+
const { socket, handlers, toEmit } = createSocket()
155+
const roomManager = createRoomManager({
156+
getRoomForSocket: vi.fn().mockResolvedValue(TABLE_ROOM),
157+
})
158+
setupTablesHandlers(socket as unknown as SetupArg, roomManager)
159+
160+
const cell = {
161+
anchor: { rowId: 'row-1', columnId: 'col-a' },
162+
focus: { rowId: 'row-1', columnId: 'col-a' },
163+
editing: true,
164+
}
165+
await handlers[TABLE_PRESENCE_EVENTS.CELL_SELECTION]({ cell })
166+
167+
expect(roomManager.updateUserActivity).toHaveBeenCalledWith(TABLE_ROOM, 'socket-1', { cell })
168+
// Namespaced room → broadcast targets roomName(room), not the bare id.
169+
expect(socket.to).toHaveBeenCalledWith('table:table-1')
170+
// The delta carries only the socket id + cell — identity comes from the roster.
171+
expect(toEmit).toHaveBeenCalledWith(TABLE_PRESENCE_EVENTS.CELL_SELECTION, {
172+
socketId: 'socket-1',
173+
cell,
174+
})
175+
})
176+
177+
it('drops a malformed cell selection without storing or relaying it', async () => {
178+
const { socket, handlers, toEmit } = createSocket()
179+
const roomManager = createRoomManager({
180+
getRoomForSocket: vi.fn().mockResolvedValue(TABLE_ROOM),
181+
})
182+
setupTablesHandlers(socket as unknown as SetupArg, roomManager)
183+
184+
await handlers[TABLE_PRESENCE_EVENTS.CELL_SELECTION]({ cell: { anchor: 'x"]' } })
185+
186+
expect(roomManager.updateUserActivity).not.toHaveBeenCalled()
187+
expect(toEmit).not.toHaveBeenCalled()
188+
})
189+
190+
it('aborts a stale join whose authorize resolves after a newer join', async () => {
191+
const { socket, handlers } = createSocket()
192+
const roomManager = createRoomManager()
193+
// First join's authorize hangs until released; the second resolves immediately.
194+
let releaseA: (value: unknown) => void = () => {}
195+
const pendingA = new Promise((resolve) => {
196+
releaseA = resolve
197+
})
198+
mockAuthorizeRoom.mockReturnValueOnce(pendingA).mockResolvedValue({
199+
allowed: true,
200+
status: 200,
201+
workspaceId: 'ws-1',
202+
workspacePermission: 'admin',
203+
})
204+
setupTablesHandlers(socket as unknown as SetupArg, roomManager)
205+
206+
const joinA = handlers[TABLE_PRESENCE_EVENTS.JOIN]({ tableId: 'table-A' })
207+
await handlers[TABLE_PRESENCE_EVENTS.JOIN]({ tableId: 'table-B' })
208+
releaseA({ allowed: true, status: 200, workspaceId: 'ws-1', workspacePermission: 'admin' })
209+
await joinA
210+
211+
// The newer join (B) wins; the stale A join aborts before touching room state.
212+
expect(socket.join).toHaveBeenCalledWith('table:table-B')
213+
expect(socket.join).not.toHaveBeenCalledWith('table:table-A')
214+
expect(roomManager.addUserToRoom).not.toHaveBeenCalledWith(
215+
{ type: ROOM_TYPES.TABLE, id: 'table-A' },
216+
expect.anything(),
217+
expect.anything()
218+
)
219+
})
220+
221+
it('leaves the table room on leave', async () => {
222+
const { socket, handlers } = createSocket()
223+
const roomManager = createRoomManager({
224+
getRoomForSocket: vi.fn().mockResolvedValue(TABLE_ROOM),
225+
})
226+
setupTablesHandlers(socket as unknown as SetupArg, roomManager)
227+
228+
await handlers[TABLE_PRESENCE_EVENTS.LEAVE]({ tableId: 'table-1' })
229+
230+
expect(socket.leave).toHaveBeenCalledWith('table:table-1')
231+
expect(roomManager.removeUserFromRoom).toHaveBeenCalledWith(TABLE_ROOM, 'socket-1')
232+
expect(roomManager.broadcastPresenceUpdate).toHaveBeenCalledWith(TABLE_ROOM, 'socket-1')
233+
})
234+
})

0 commit comments

Comments
 (0)