feat(agent): allow variable tool permission modes by j15z · Pull Request #7538 · simstudioai/sim

j15z · 2026-09-05T23:29:55Z

Summary

Let each agent tool's Permission Mode use a fixed selector or a variable such as <start.toolMode>. Resolve variables before filtering tools and reject invalid modes before calling the provider.
Put Permission Mode in the expanded tool parameters with the canonical arrow toggle. Make both modes available without a feature flag across the editor, API/CLI, Copilot, and imports.
Preserve the fixed choice, variable input, and active mode through search, export/import, and tool-array changes. Save reordered tools and their mode indexes in one collaborative operation.
Keep disabled tools out of discovery/execution, preserve secret tracking during resolution, and retain locked-block protections. Keep edits arriving during pending saves separate and compare search replacements independently of object key order.
Report interrupted CLI responses with guidance for uncertain writes, and show import failures in the editor.

Companion: simstudioai/mothership#488

Type of Change

Testing

Latest changes: 502 application regression tests and all 389 realtime tests pass. Coverage includes fixed/variable resolution, invalid values, secret provenance, API/Copilot edits and dry runs, imports, search, atomic reordering, locked containers, and edits arriving during pending or failed database writes.
App/realtime type checks, repository lint, all 46 audits, API validation, block-registry, and docs-manifest checks pass. Regenerated committed artifacts and verified the removed flag and policy endpoint have no remaining source references.
Earlier Chrome/CLI/API testing covered Auto/Force/None, all five tool shapes, dynamic inputs, workflow variables, secrets, empty lists, reload, cross-tab editing, deployed/draft isolation, rollback, and 100-tool reversal/deletion.
Earlier fault testing covered scoped scheduled-job execution/recovery, dropped requests and responses around commits, injected 503s, tool cancellation, and recovery after a failed tool call.
Full CLI suite previously passed 990 tests with one skip and two failures in unchanged update-check process tests: Node emitted a proxy warning where those tests require empty stderr. Changed HTTP-client tests passed.

Remaining coverage: the latest simplification has automated coverage but was not rerun in Chrome. Undo/redo, additional collaborative/network interleavings, viewer-account behavior, live LLM-provider outage injection, global cron ticks, and queue transport remain unverified. The existing multi-request importer is not transactional for arbitrary network failures.

Review focus: keeping each tool's permission attached through array edits, excluding disabled tools from discovery/execution, and preserving both selector and variable inputs across workflow operations.

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Screenshots/Videos

No screenshots or videos attached.

vercel · 2026-09-05T23:30:02Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Sep 10, 2026 12:28am UTC

j15z · 2026-09-06T00:02:56Z

@greptile

j15z · 2026-09-06T00:03:01Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-06T00:03:07Z

@cubic-dev-ai review this PR

@j15z I have started the AI code review. It will take a few minutes to complete.

greptile-apps · 2026-09-06T00:06:48Z

Confidence Score: 5/5

The PR appears safe to merge; no new actionable defects remain, and all previous findings are resolved or no longer applicable.

Summary

Resolves variable permission expressions before filtering tools and rejects invalid resolved modes before provider execution.
Preserves tool permission settings and canonical-mode indexes across array edits and atomic realtime writes.
Excludes disabled tools from discovery and execution.
Improves workflow import and interrupted CLI-response error reporting.
Adds regression coverage across executor, realtime, workflow editing, persistence, sanitization, search, API contracts, and CLI behavior.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Editor[Workflow editor] --> Mode[Fixed or variable permission mode]
  Mode --> Persist[Workflow and realtime persistence]
  Persist --> Resolve[Execution-time variable resolution]
  Resolve --> Validate{Auto, Force, or None?}
  Validate -->|Invalid| Reject[Reject before provider call]
  Validate -->|None| Exclude[Exclude tool]
  Validate -->|Auto or Force| Provider[Provider execution]
  Persist --> Export[API, CLI, export and import]

_{Reviews (7) · Last reviewed commit: "improvement(agent): remove permission mo..."}

github-actions · 2026-09-09T20:33:45Z

⚠️ Cross-repo companion check

One or more companion PRs aren't merged into staging yet. Merging this without them will leave copilot and sim out of sync — merge them in lockstep.

❌ simstudioai/mothership#488 — OPEN, not merged (targets staging) — feat(workflow): document variable agent tool permissions

j15z · 2026-09-09T23:46:19Z

@greptile

j15z · 2026-09-09T23:46:20Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-09T23:46:26Z

@cubic-dev-ai review this PR

@j15z I have started the AI code review. It will take a few minutes to complete.

j15z · 2026-09-10T00:03:44Z

@greptile

j15z · 2026-09-10T00:03:45Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-10T00:03:53Z

@cubic-dev-ai review this PR

@j15z I have started the AI code review. It will take a few minutes to complete.

j15z · 2026-09-10T00:05:23Z

@greptile

j15z · 2026-09-10T00:05:25Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-10T00:05:30Z

@cubic-dev-ai review this PR

@j15z I have started the AI code review. It will take a few minutes to complete.

j15z · 2026-09-10T00:13:25Z

@greptile

j15z · 2026-09-10T00:13:26Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-10T00:13:33Z

@cubic-dev-ai review this PR

@j15z I have started the AI code review. It will take a few minutes to complete.

j15z · 2026-09-10T00:26:35Z

@greptile

j15z · 2026-09-10T00:26:36Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-10T00:26:41Z

@cubic-dev-ai review this PR

@j15z I have started the AI code review. It will take a few minutes to complete.

j15z · 2026-09-10T00:31:18Z

@greptile

j15z · 2026-09-10T00:31:19Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-10T00:31:25Z

@cubic-dev-ai review this PR

@j15z I have started the AI code review. It will take a few minutes to complete.

vercel Bot temporarily deployed to Preview September 5, 2026 23:54 Inactive

greptile-apps Bot reviewed Sep 6, 2026

View reviewed changes

j15z force-pushed the codex/agent-tool-permission-mode branch from 11c3d58 to 818dca4 Compare September 6, 2026 00:21

vercel Bot temporarily deployed to Preview September 6, 2026 00:21 Inactive

j15z changed the base branch from codex/canonical-tool-mode-base to staging September 6, 2026 00:22

j15z force-pushed the codex/agent-tool-permission-mode branch from 818dca4 to 41af1d3 Compare September 7, 2026 19:00

vercel Bot deployed to Preview September 7, 2026 19:02 View deployment

j15z added 4 commits September 9, 2026 13:25

feat(agent): add variable permission modes behind a feature flag

0d54173

fix(agent): gate variable permission mode workflow writes

61c6130

fix(agent): use the existing canonical permission mode toggle

4643393

improvement(agent): align permission controls with shared components

661f655

j15z force-pushed the codex/agent-tool-permission-mode branch from 41af1d3 to 661f655 Compare September 9, 2026 20:30

vercel Bot deployed to Preview September 9, 2026 20:33 View deployment

github-actions Bot added the requires-mothership-merge Has a companion PR on the mothership/copilot side — merge in lockstep label Sep 9, 2026

fix(agent): harden permission editing and failure recovery

22c859e

vercel Bot temporarily deployed to Preview September 9, 2026 21:56 Inactive

fix(agent): gate realtime tool permission writes

1316299

vercel Bot deployed to Preview September 9, 2026 23:47 View deployment

greptile-apps Bot reviewed Sep 9, 2026

View reviewed changes

Comment thread apps/realtime/src/database/operations.ts Outdated

Comment thread apps/realtime/src/database/workflow-authoring.ts Outdated

fix(agent): preserve permissions across gated workflow edits

68451c9

j15z force-pushed the codex/agent-tool-permission-mode branch from d433d88 to 68451c9 Compare September 10, 2026 00:04

vercel Bot deployed to Preview September 10, 2026 00:06 View deployment

greptile-apps Bot reviewed Sep 10, 2026

View reviewed changes

Comment thread apps/sim/lib/workflows/persistence/utils.ts

Comment thread packages/workflow-types/src/agent-tool-permissions.ts Outdated

fix(agent): lock external saves and distinguish workflow targets

82e55e9

vercel Bot deployed to Preview September 10, 2026 00:14 View deployment

improvement(agent): remove permission mode feature flag

fea4067

vercel Bot deployed to Preview September 10, 2026 00:28 View deployment

greptile-apps Bot reviewed Sep 10, 2026

View reviewed changes

Comment thread ...ponents/editor/components/sub-block/components/tool-input/components/tools/usage-control.tsx

Uh oh!

Conversation

j15z commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

Testing

Checklist

Screenshots/Videos

Uh oh!

vercel Bot commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

j15z commented Sep 6, 2026

Uh oh!

j15z commented Sep 6, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 6, 2026

Uh oh!

greptile-apps Bot commented Sep 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Diagram

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Sep 9, 2026

⚠️ Cross-repo companion check

Uh oh!

j15z commented Sep 9, 2026

Uh oh!

j15z commented Sep 9, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 9, 2026

Uh oh!

Uh oh!

Uh oh!

j15z commented Sep 10, 2026

Uh oh!

j15z commented Sep 10, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 10, 2026

Uh oh!

j15z commented Sep 10, 2026

Uh oh!

j15z commented Sep 10, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 10, 2026

Uh oh!

Uh oh!

Uh oh!

j15z commented Sep 10, 2026

Uh oh!

j15z commented Sep 10, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 10, 2026

Uh oh!

j15z commented Sep 10, 2026

Uh oh!

j15z commented Sep 10, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 10, 2026

Uh oh!

Uh oh!

j15z commented Sep 10, 2026

Uh oh!

j15z commented Sep 10, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

j15z commented Sep 5, 2026 •

edited

Loading

vercel Bot commented Sep 5, 2026 •

edited

Loading

greptile-apps Bot commented Sep 6, 2026 •

edited

Loading