Skip to content

Commit f76aca1

Browse files
waleedlatif1waleedlatif1
authored
feat(files): workspace file version history over the v2 API, CLI, and MCP (#7997)
* feat(files): workspace file version history over the v2 API, CLI, and MCP * fix(files): address review feedback on file version history * chore(db): regenerate file version migration after staging's 0364 * improvement(files): scope version rows to the write's workspace and narrow stored provenance status * fix(files): release purged file history atomically with restore * fix(files): release file history through the outbox after the current object is gone * fix(files): release purged history inside the file-row purge transaction * fix(files): resolve metadata version from the record's own storage key * fix(files): never pair a stale file record with a newer version number * improvement(files): simplify file version history internals - read metadata and its current version in one statement; drop the retry and 409 - move provenance policy branching into the provenance module - project stored provenance out of list, head, and get reads; revert reads it on demand - chunk storage-cleanup enqueues to the outbox bulk limit in one place - drop the write-only content_updated_at version column (unreleased 0365) - reuse findCause, the shared cleanup batch constants, and the version-number primitives - share the v2 text presenter between the file and version routes
1 parent 5cb88fd commit f76aca1

83 files changed

Lines changed: 34027 additions & 391 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/test-build.yml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,12 @@ jobs:
149149
KNOWLEDGE_ACL_TEST_DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_auth_scim
150150
run: bunx vitest run --mode integration lib/workspace-files/search/dispatcher.integration.ts
151151

152+
- name: Verify workspace file version history on PostgreSQL 17
153+
working-directory: apps/sim
154+
env:
155+
KNOWLEDGE_ACL_TEST_DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_auth_scim
156+
run: bunx vitest run --mode integration lib/uploads/contexts/workspace/__integration__/file-versions.integration.ts
157+
152158
- name: Verify file search trigram estimate against pg_trgm
153159
working-directory: apps/sim
154160
env:

‎apps/docs/content/docs/cli/files.mdx‎

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -174,6 +174,163 @@ sim files delete <fileId> [options]
174174

175175
</CommandTable>
176176

177+
## Permanently delete a previous version of a file
178+
179+
```bash
180+
sim files versions delete <fileId> <version> [options]
181+
```
182+
183+
**Arguments**
184+
185+
<CommandTable>
186+
187+
| Argument | Required | Description |
188+
| --- | --- | --- |
189+
| `fileId` | Yes | File identifier. |
190+
| `version` | Yes | Version number. |
191+
192+
</CommandTable>
193+
194+
**Options**
195+
196+
<CommandTable>
197+
198+
| Option | Required | Description |
199+
| --- | --- | --- |
200+
| `-y, --yes` | Yes | Confirm this operation. |
201+
202+
</CommandTable>
203+
204+
## Show the metadata of one version of a file
205+
206+
```bash
207+
sim files versions describe <fileId> <version>
208+
```
209+
210+
**Arguments**
211+
212+
<CommandTable>
213+
214+
| Argument | Required | Description |
215+
| --- | --- | --- |
216+
| `fileId` | Yes | File identifier. |
217+
| `version` | Yes | Version number. |
218+
219+
</CommandTable>
220+
221+
## List the recorded versions of a file
222+
223+
```bash
224+
sim files versions list <fileId> [options]
225+
```
226+
227+
**Arguments**
228+
229+
<CommandTable>
230+
231+
| Argument | Required | Description |
232+
| --- | --- | --- |
233+
| `fileId` | Yes | File identifier. |
234+
235+
</CommandTable>
236+
237+
**Options**
238+
239+
<CommandTable>
240+
241+
| Option | Required | Description |
242+
| --- | --- | --- |
243+
| `--sort-by <value>` | No | Field used to sort the result. Accepted values: `version`. |
244+
| `--sort-order <value>` | No | Sort direction. Accepted values: `asc`, `desc`. |
245+
| `--limit <n>` | No | Maximum items to return (0 for everything). Defaults to `100`. |
246+
| `--cursor <value>` | No | Continue from nextCursor returned by a previous result. |
247+
248+
</CommandTable>
249+
250+
## Read the text content of one version of a file
251+
252+
```bash
253+
sim files versions read <fileId> <version> [options]
254+
```
255+
256+
**Arguments**
257+
258+
<CommandTable>
259+
260+
| Argument | Required | Description |
261+
| --- | --- | --- |
262+
| `fileId` | Yes | File identifier. |
263+
| `version` | Yes | Version number. |
264+
265+
</CommandTable>
266+
267+
**Options**
268+
269+
<CommandTable>
270+
271+
| Option | Required | Description |
272+
| --- | --- | --- |
273+
| `--max-bytes <value>` | No | Optional ceiling on the source bytes fed to the parser, lowering but never raising the server limit. |
274+
| `--offset <value>` | No | First line to return, 1-based. Absent starts at the first line. |
275+
| `--limit <value>` | No | How many lines to return from `offset`. Absent reads to the end. |
276+
277+
</CommandTable>
278+
279+
## Make a previous version of a file current again
280+
281+
```bash
282+
sim files versions revert <fileId> <version> [options]
283+
```
284+
285+
**Arguments**
286+
287+
<CommandTable>
288+
289+
| Argument | Required | Description |
290+
| --- | --- | --- |
291+
| `fileId` | Yes | File identifier. |
292+
| `version` | Yes | Version number. |
293+
294+
</CommandTable>
295+
296+
**Options**
297+
298+
<CommandTable>
299+
300+
| Option | Required | Description |
301+
| --- | --- | --- |
302+
| `--expected-current-version <value>` | No | Revert only while this is still the current version; otherwise the request fails with `409`. Omit to revert whatever is current. Collaborative edits and repeated workflow writes that fold into the current version keep its number. |
303+
304+
</CommandTable>
305+
306+
## Download the content of one version of a file
307+
308+
```bash
309+
sim files versions download <fileId> <version> [options]
310+
```
311+
312+
**Arguments**
313+
314+
<CommandTable>
315+
316+
| Argument | Required | Description |
317+
| --- | --- | --- |
318+
| `fileId` | Yes | File identifier. |
319+
| `version` | Yes | Version number. |
320+
321+
</CommandTable>
322+
323+
**Options**
324+
325+
<CommandTable>
326+
327+
| Option | Required | Description |
328+
| --- | --- | --- |
329+
| `-o, --output-file <path>` | No | Write content to a file instead of stdout. |
330+
| `--force` | No | Overwrite --output-file if it already exists. |
331+
332+
</CommandTable>
333+
177334
## Apply one exact or anchor-based edit to a text file
178335

179336
```bash

‎apps/docs/content/docs/cli/reference.mdx‎

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -909,6 +909,175 @@ sim files delete <fileId> [options]
909909

910910
</CommandTable>
911911

912+
### sim files versions delete
913+
914+
Permanently delete a previous version of a file
915+
916+
```bash
917+
sim files versions delete <fileId> <version> [options]
918+
```
919+
920+
**Arguments**
921+
922+
<CommandTable>
923+
924+
| Argument | Required | Description |
925+
| --- | --- | --- |
926+
| `fileId` | Yes | File identifier. |
927+
| `version` | Yes | Version number. |
928+
929+
</CommandTable>
930+
931+
**Options**
932+
933+
<CommandTable>
934+
935+
| Option | Required | Description |
936+
| --- | --- | --- |
937+
| `-y, --yes` | Yes | Confirm this operation. |
938+
939+
</CommandTable>
940+
941+
### sim files versions describe
942+
943+
Show the metadata of one version of a file
944+
945+
```bash
946+
sim files versions describe <fileId> <version>
947+
```
948+
949+
**Arguments**
950+
951+
<CommandTable>
952+
953+
| Argument | Required | Description |
954+
| --- | --- | --- |
955+
| `fileId` | Yes | File identifier. |
956+
| `version` | Yes | Version number. |
957+
958+
</CommandTable>
959+
960+
### sim files versions list
961+
962+
List the recorded versions of a file
963+
964+
```bash
965+
sim files versions list <fileId> [options]
966+
```
967+
968+
**Arguments**
969+
970+
<CommandTable>
971+
972+
| Argument | Required | Description |
973+
| --- | --- | --- |
974+
| `fileId` | Yes | File identifier. |
975+
976+
</CommandTable>
977+
978+
**Options**
979+
980+
<CommandTable>
981+
982+
| Option | Required | Description |
983+
| --- | --- | --- |
984+
| `--sort-by <value>` | No | Field used to sort the result. Accepted values: `version`. |
985+
| `--sort-order <value>` | No | Sort direction. Accepted values: `asc`, `desc`. |
986+
| `--limit <n>` | No | Maximum items to return (0 for everything). Defaults to `100`. |
987+
| `--cursor <value>` | No | Continue from nextCursor returned by a previous result. |
988+
989+
</CommandTable>
990+
991+
### sim files versions read
992+
993+
Read the text content of one version of a file
994+
995+
```bash
996+
sim files versions read <fileId> <version> [options]
997+
```
998+
999+
**Arguments**
1000+
1001+
<CommandTable>
1002+
1003+
| Argument | Required | Description |
1004+
| --- | --- | --- |
1005+
| `fileId` | Yes | File identifier. |
1006+
| `version` | Yes | Version number. |
1007+
1008+
</CommandTable>
1009+
1010+
**Options**
1011+
1012+
<CommandTable>
1013+
1014+
| Option | Required | Description |
1015+
| --- | --- | --- |
1016+
| `--max-bytes <value>` | No | Optional ceiling on the source bytes fed to the parser, lowering but never raising the server limit. |
1017+
| `--offset <value>` | No | First line to return, 1-based. Absent starts at the first line. |
1018+
| `--limit <value>` | No | How many lines to return from `offset`. Absent reads to the end. |
1019+
1020+
</CommandTable>
1021+
1022+
### sim files versions revert
1023+
1024+
Make a previous version of a file current again
1025+
1026+
```bash
1027+
sim files versions revert <fileId> <version> [options]
1028+
```
1029+
1030+
**Arguments**
1031+
1032+
<CommandTable>
1033+
1034+
| Argument | Required | Description |
1035+
| --- | --- | --- |
1036+
| `fileId` | Yes | File identifier. |
1037+
| `version` | Yes | Version number. |
1038+
1039+
</CommandTable>
1040+
1041+
**Options**
1042+
1043+
<CommandTable>
1044+
1045+
| Option | Required | Description |
1046+
| --- | --- | --- |
1047+
| `--expected-current-version <value>` | No | Revert only while this is still the current version; otherwise the request fails with `409`. Omit to revert whatever is current. Collaborative edits and repeated workflow writes that fold into the current version keep its number. |
1048+
1049+
</CommandTable>
1050+
1051+
### sim files versions download
1052+
1053+
Download the content of one version of a file
1054+
1055+
```bash
1056+
sim files versions download <fileId> <version> [options]
1057+
```
1058+
1059+
**Arguments**
1060+
1061+
<CommandTable>
1062+
1063+
| Argument | Required | Description |
1064+
| --- | --- | --- |
1065+
| `fileId` | Yes | File identifier. |
1066+
| `version` | Yes | Version number. |
1067+
1068+
</CommandTable>
1069+
1070+
**Options**
1071+
1072+
<CommandTable>
1073+
1074+
| Option | Required | Description |
1075+
| --- | --- | --- |
1076+
| `-o, --output-file <path>` | No | Write content to a file instead of stdout. |
1077+
| `--force` | No | Overwrite --output-file if it already exists. |
1078+
1079+
</CommandTable>
1080+
9121081
### sim files edit
9131082

9141083
Apply one exact or anchor-based edit to a text file

‎apps/docs/content/docs/platform/enterprise/data-retention.mdx‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,12 @@ Controls how long **Chat data** is kept, including:
7171
- Run checkpoints and async tool calls
7272
- Inbox tasks
7373

74+
### Previous file versions
75+
76+
Every change to a file's content keeps the previous content as a version you can read or revert to through the API, CLI, or MCP server. This setting controls how long a version is kept after a newer one replaces it. The newest ten versions of each file are always kept, whatever their age.
77+
78+
Without a setting, previous versions are kept until a file reaches 500 of them. The setting isn't on the settings page yet: set `fileVersionRetentionHours` through the data retention API, either for the organization or in a workspace override.
79+
7480
Each setting is independent. You can configure a short log retention period alongside a long soft deletion cleanup period, or any combination that fits your compliance requirements.
7581

7682
---
@@ -186,16 +192,17 @@ Once enabled, retention settings are configurable through **Settings → Organiz
186192

187193
### Scheduling the deletion pass
188194

189-
`DATA_RETENTION_ENABLED` permits deletion; it does not perform it. Deletion runs when a scheduled request reaches one of three endpoints, each authenticated with a bearer token equal to `CRON_SECRET`:
195+
`DATA_RETENTION_ENABLED` permits deletion; it does not perform it. Deletion runs when a scheduled request reaches one of four endpoints, each authenticated with a bearer token equal to `CRON_SECRET`:
190196

191197
| Category | Endpoint |
192198
|----------|----------|
193199
| Execution and job logs | `GET /api/logs/cleanup` |
194200
| Soft-deleted resources | `GET /api/cron/cleanup-soft-deletes` |
195201
| Chats and Chat runs | `GET /api/cron/cleanup-tasks` |
202+
| Previous file versions | `GET /api/cron/cleanup-file-versions` |
196203

197204
<Callout type="warn">
198-
Neither shipped deployment schedules these three endpoints — not the Helm chart, not Docker Compose's `cron` service. An operator who sets `DATA_RETENTION_ENABLED=true` alone still deletes nothing. Add them to `cronjobs.jobs`, or call them daily from an external scheduler.
205+
Neither shipped deployment schedules these four endpoints — not the Helm chart, not Docker Compose's `cron` service. An operator who sets `DATA_RETENTION_ENABLED=true` alone still deletes nothing. Add them to `cronjobs.jobs`, or call them daily from an external scheduler.
199206
</Callout>
200207

201208
```bash

‎apps/docs/content/docs/platform/enterprise/self-hosted.mdx‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -86,10 +86,11 @@ Persist that value as `CRON_SECRET` on the app **and** on whatever calls these e
8686
| Retention — logs | `GET /api/logs/cleanup` | Daily | **No** — schedule it yourself |
8787
| Retention — soft deletes | `GET /api/cron/cleanup-soft-deletes` | Daily | **No** — schedule it yourself |
8888
| Retention — Chat tasks | `GET /api/cron/cleanup-tasks` | Daily | **No** — schedule it yourself |
89+
| Retention — file versions | `GET /api/cron/cleanup-file-versions` | Daily | **No** — schedule it yourself |
8990
| OAuth token cleanup | `GET /api/cron/cleanup-oauth-tokens` | Hourly | Yes — Helm and Docker Compose both call it |
9091

9192
<Callout type="warn">
92-
Both shipped deployments schedule the data-drain dispatcher and OAuth token cleanup, but **not** the three configurable data-retention endpoints. Setting `DATA_RETENTION_ENABLED=true` alone deletes no retained product data — those windows are evaluated only when one of the three endpoints is called. Add them to `cronjobs.jobs` yourself, or drive them from an external scheduler.
93+
Both shipped deployments schedule the data-drain dispatcher and OAuth token cleanup, but **not** the four configurable data-retention endpoints. Setting `DATA_RETENTION_ENABLED=true` alone deletes no retained product data — those windows are evaluated only when one of the four endpoints is called. Add them to `cronjobs.jobs` yourself, or drive them from an external scheduler.
9394

9495
OAuth token cleanup runs independently of sign-in activity, removing expired and revoked credentials. See [Sign in with Sim](/platform/self-hosting/authentication#sign-in-with-sim) for provider configuration.
9596
</Callout>

0 commit comments

Comments
 (0)