feat(coder-templates/ai-agent-generic): add opt-in virtual desktop for computer_use#55
Open
ausbru87 wants to merge 1 commit into
Open
feat(coder-templates/ai-agent-generic): add opt-in virtual desktop for computer_use#55ausbru87 wants to merge 1 commit into
ausbru87 wants to merge 1 commit into
Conversation
…r computer_use The computer_use subagent (spawn_agent type=computer_use) drives a graphical desktop through the Coder agent's desktop endpoints, which shell out to a self-contained portabledesktop binary resolved from the agent script bin dir. Without that binary the desktop session fails to start, so computer_use cannot run in workspaces from this template. Add an enable_desktop parameter (default on) that installs the registry.coder.com/coder/portabledesktop/coder module. The binary installs without sudo, so this keeps the hardened posture: privilege escalation stays disabled and the enterprise-base image is unchanged. The desktop process only starts when a computer_use action first connects, so at rest the only added cost is a github.com release download at startup, the same egress class as the existing code-server module. The parameter is the off-switch for a tight-egress or air-gapped boundary, where the module url can instead point at an in-boundary mirror with a pinned sha256. computer_use also requires platform-layer config not in this template: the Virtual Desktop experiment and a computer-use provider key. Generated by Coder Agents.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enables the
computer_usesubagent (spawn_agentwithtype=computer_use) inai-agent-genericworkspaces by adding an opt-inenable_desktopparameter (default on) that installs theportabledesktopregistry module.Why
computer_usedrives a graphical desktop through the Coder agent's built-in desktop endpoints (/api/v0/desktop/action). The agent implements those by shelling out to a self-containedportabledesktopbinary resolved from the agent script bin dir (agent/x/agentdesktop). Without that binary the desktop session fails to start, socomputer_usecannot run in workspaces from this template. The official way to provide it is theportabledesktopmodule.This is not KasmVNC/XFCE/xdotool: the binary bundles its own Xvnc + window manager.
What changed
main.tf: newenable_desktopbool parameter (defaulttrue) and a conditionalregistry.coder.com/coder/portabledesktop/coder0.1.0 module.README.md: Virtual desktop section, parameter row, tooling note, egress/prereq tradeoffs.The hardened posture is preserved: the binary installs without sudo, so
allow_privilege_escalationstaysfalseand theenterprise-baseimage is unchanged. The desktop process only starts when acomputer_useaction first connects.Tradeoff (flagged for review)
This template is the default fallback for all non-language-specific agent tasks.
enable_desktop=trueadds a one-timegithub.comrelease download at startup for every generic workspace (the same egress class as the existingcode-servermodule). The parameter is the off-switch for a tight-egress/air-gapped boundary, where the moduleurlcan instead point at an in-boundary mirror with a pinnedsha256. Defaulted on socomputer_useworks out of the box; happy to flip to off-by-default if preferred.computer_usealso needs platform-layer config that is not in this template: the Virtual Desktop experiment and a computer-use provider key. Both are already enabled on the demo deployment (provider = anthropic).Verification
Investigation and test evidence
coder/coder:coderd/x/chatd/chattool/computeruse.go->workspacesdk.AgentConn.ExecuteDesktopAction-> agentagent/x/agentdesktop->portabledesktopCLI (exec.LookPath/ script bin dir).terraform fmt -check,init, andvalidateall pass; module resolves toportabledesktop0.1.0. No emdash/endash.coder(active versionsmiling_satterfield57); plan resolvesenable_desktop.portabledesktop-linux-x64binary (statically linked, 54MB) inside the exact ECRenterprise-base:ubuntu-noble-20260601image as uid 1000:up --jsonstartedXvnc+openboxonvncPort 5901with no errors, andscreenshot --jsonreturned a valid PNG. This is precisely what the agent's desktop endpoints invoke.Note:
ai-agent-genericrequires a GitLab login, so a new workspace from it only finishes building after the owner completes the GitLab OAuth flow (pre-existing behavior, unchanged by this PR).Generated by Coder Agents, on behalf of @ausbru87.