Skip to content

Latest commit

 

History

History
163 lines (124 loc) · 25.5 KB

CLAUDE.md

File metadata and controls

163 lines (124 loc) · 25.5 KB

CLAUDE.md

Guidance for Claude Code / AI agents working in this repo. This file is loaded into every session — keep it lean. It is orientation + behavior only; detailed reference lives in docs/.

Project Overview

MCPProxy is a Go desktop application that acts as a smart proxy for AI agents using the Model Context Protocol (MCP): intelligent tool discovery, massive token savings, and built-in security quarantine against malicious MCP servers.

Stack: Go 1.26 (backend) · TypeScript 5.9 / Vue 3.5 (frontend) · Swift 5.9 (macOS tray). Storage: BBolt (config.db) + Bleve (search index). Avoid new dependencies without clear need.

Editions (Personal & Server)

Built in two editions from one codebase via Go build tags:

Edition Build Binary Distribution
Personal (default) go build ./cmd/mcpproxy mcpproxy macOS DMG, Windows installer, Linux tar.gz
Server go build -tags server -o mcpproxy-server ./cmd/mcpproxy mcpproxy-server Docker image (ghcr.io) only — no .deb / tar.gz

All server code is behind //go:build server in internal/serveredition/; the personal edition is unaffected. The binary self-identifies (mcpproxy version, /api/v1/status → "edition"). Server multi-user OAuth (Spec 024): see docs/development/server-edition-multiuser-auth.md.

Every feature decision should ask: "Does this make the personal edition so good that developers tell their teammates about it?"

Architecture

Core + Tray split: mcpproxy (headless HTTP API + MCP proxy) and mcpproxy-tray (GUI that manages the core). The tray is a UI controller — it holds no state; it reads/writes core config via REST + SSE. Tray↔core over a Unix socket (~/.mcpproxy/mcpproxy.sock) / named pipe on Windows; socket connections bypass the API key (OS-level auth), TCP requires it.

Directory Purpose
cmd/mcpproxy/ CLI entry point (Cobra)
cmd/mcpproxy-tray/ System tray app (state machine)
internal/runtime/ Lifecycle, event bus, background services
internal/server/ HTTP server, MCP proxy
internal/httpapi/ REST API (/api/v1)
internal/upstream/ 3-layer client: core/managed/cli
internal/config/ Configuration management
internal/index/ Bleve BM25 search index
internal/storage/ BBolt database
internal/oauth/ OAuth 2.1 + PKCE
internal/security/ Sensitive-data detection + quarantine
internal/serveredition/ Server-only code (//go:build server)
native/macos/MCPProxy/ Swift macOS tray app

See docs/architecture.md and docs/socket-communication.md.

Development Commands

# Build
go build -o mcpproxy ./cmd/mcpproxy                       # core (personal)
go build -tags server -o mcpproxy-server ./cmd/mcpproxy   # core (server edition)
make build                                                # frontend + backend
make build-docker                                         # server Docker image

# Test — ALWAYS run before committing
./scripts/test-api-e2e.sh                                 # quick API E2E (required)
go test -race ./internal/... -v                           # unit + race
go test -tags server ./internal/serveredition/... -race   # server edition
./scripts/run-all-tests.sh                                # full suite

# Lint — CI uses golangci-lint v2 with .github/.golangci.yml, which is STRICTER
# than the local scripts/run-linter.sh (v1.x) and catches things it misses.
# CI runs it TWICE: bare, and with --build-tags server (server-edition code is
# invisible to the bare run). Run both before pushing:
/opt/homebrew/bin/golangci-lint run --config .github/.golangci.yml ./...
/opt/homebrew/bin/golangci-lint run --config .github/.golangci.yml --build-tags server ./...
# CI also race-tests internal/server, httpapi and storage under -tags server
# with the unit-tests.yml -skip regex (bare `go test ./internal/server/...`
# hangs to the timeout on the binary-spawning tests):
go test -race -tags server -timeout 20m -skip "E2E|Binary|MCPProtocol|TestInfoEndpoint|TestGracefulShutdownNoPanic|TestSocketInfoEndpoint" ./internal/serveredition/... ./internal/config/... ./internal/oauth/... ./internal/server/... ./internal/httpapi/... ./internal/storage/...

# Run
./mcpproxy serve [--listen :8080] [--log-level=debug]     # core (localhost:8080)
./mcpproxy-tray                                           # tray (auto-starts core)

CLI management — mcpproxy upstream|tools|activity|token|telemetry|feedback|doctor|update … (update is channel-aware: guidance for package-manager installs, verified self-update only on tarball). Output: -o json|yaml, MCPPROXY_OUTPUT=json, --help-json (machine-readable for agents). References: docs/cli-management-commands.md · docs/cli/activity-commands.md · docs/features/agent-tokens.md · docs/cli-output-formatting.md.

Verifying Web-UI changes (Playwright sweep + HTML report) — required when touching frontend/src/: docs/development/web-ui-verification.md.

Configuration

Default locations: Config ~/.mcpproxy/mcp_config.json · Data ~/.mcpproxy/config.db (BBolt) · Index ~/.mcpproxy/index.bleve/ · Logs ~/.mcpproxy/logs/.

{
  "listen": "127.0.0.1:8080",
  "api_key": "auto-generated-if-empty",
  "require_mcp_auth": false,
  "enable_socket": true,
  "enable_web_ui": true,
  "mcpServers": [
    { "name": "github", "url": "https://api.github.com/mcp", "protocol": "http", "enabled": true },
    { "name": "ast-grep", "command": "npx", "args": ["ast-grep-mcp"], "working_dir": "/path", "protocol": "stdio", "enabled": true }
  ]
}

Env vars: MCPPROXY_LISTEN, MCPPROXY_API_KEY, MCPPROXY_DEBUG, MCPPROXY_TELEMETRY=false, HEADLESS. Full reference: docs/configuration.md.

MCP Protocol

Built-in tools: retrieve_tools (BM25 search across upstream tools; Spec 049 opt-in include_disabled; Spec 085 detail override + compact signatures under tool_response_mode: compact) · describe_tool (Spec 085: batch ≤5 ids → full schemas; Spec 102: also on the direct surface, server:tool or server__tool ids) · call_tool_read|write|destructive (Spec 018 intent variants; operation type inferred from the variant; Spec 085 pre-dispatch arg validation with self-healing invalid_params errors) · code_execution (sandboxed JS, on by default since v0.66.0) · upstream_servers (CRUD, Spec 049) · quarantine_security (Spec 032) · profiles (Spec 108 admin tool: profile CRUD, client bindings, explain). Tool format: <serverName>:<toolName> (e.g. github:create_issue); the direct surface lists as <serverName>__<toolName> and accepts both.

REST API base /api/v1, auth via X-API-Key header or ?apikey=. MCP endpoints (/mcp) stay unprotected for client compatibility; the REST API always requires a key (auto-generated if absent). All responses carry X-Request-Id (correlate with mcpproxy activity list --request-id <id>). Live updates via SSE at /events. Full endpoint list: oas/swagger.yaml + docs/api/rest-api.md.

All server responses include a unified health field: level (healthy|degraded|unhealthy), admin_state (enabled|disabled|quarantined), plus summary/detail/action.

Connect payload (Spec 075): GET /api/v1/connect is content-read-free (stat-only; no macOS App-Data prompt) — each ClientStatus carries access_state="unknown". GET /api/v1/connect/{client} resolves it on-demand to accessible|absent|malformed|denied (+ remediation when denied); a denied connect/disconnect returns 403 with remediation. See docs/api/rest-api.md.

Security Model

  • Localhost-only by default (127.0.0.1:8080); API key always required (auto-generated and persisted if not provided).
  • Agent tokens: scoped credentials for AI agents (mcp_agt_ prefix, HMAC-SHA256 hashed). See docs/features/agent-tokens.md.
  • Quarantine: new servers quarantined until approved; Tool Poisoning Attack (TPA) detection on descriptions. Tool-level quarantine (Spec 032): SHA-256 hashes detect new ("pending") and changed ("changed", rug-pull) tools. Trusted (non-quarantined) servers auto-approve their current toolset as a baseline only after a clean offline TPA scan of it (flagged tools land pending with the findings as hold evidence; R2-1); post-baseline changes/additions are reviewed unless per-server auto_approve_tool_changes:true (MCP-2931, deprecates skip_quarantine). Config: quarantine_enabled (global), auto_approve_tool_changes (per-server). See docs/features/security-quarantine.md.
  • require_mcp_auth: when enabled, /mcp rejects unauthenticated requests (default off, for back-compat).
  • Sensitive-data detection (internal/security/): scans tool args/responses for secrets (cloud creds, private keys, API tokens, DB strings, Luhn-validated cards, sensitive file paths, high-entropy strings). On by default; integrates with the activity log. Config under sensitive_data_detection. See docs/features/sensitive-data-detection.md.

Key Implementation Details

  • Docker isolation: runtime detection (uvx→Python, npx→Node), image selection, container lifecycle. docs/docker-isolation.md
  • OAuth: dynamic port allocation, RFC 8252 + PKCE, internal/oauth/coordinator.go, automatic token refresh. docs/oauth-resource-autodetect.md
  • Code execution: sandboxed JavaScript (ES2020+) orchestrating multiple upstream tools in one request. docs/code_execution/overview.md
  • Connection management: exponential backoff; state machine Disconnected → Connecting → Authenticating → Ready.
  • Tool indexing: full rebuild on server changes, hash-based change detection, background indexing.
  • Tool-level quarantine (Spec 032) key files: internal/storage/models.go & bbolt.go, internal/runtime/tool_quarantine.go, internal/runtime/lifecycle.go (applyDifferentialToolUpdate), internal/server/mcp.go, internal/config/config.go, frontend/src/views/ServerDetail.vue.
  • Signal handling: graceful shutdown, context cancellation, Docker cleanup, double-shutdown protection. Before running the core, kill existing instances — it locks the DB.

Debugging

mcpproxy doctor                           # quick diagnostics
mcpproxy upstream list                    # server status
mcpproxy upstream logs <name> --follow    # per-server logs
tail -f ~/Library/Logs/mcpproxy/main.log  # main log (macOS; Linux: ~/.mcpproxy/logs/main.log)

Exit codes: 0 success · 1 general · 2 port conflict · 3 DB locked · 4 config · 5 permission · 6 shutdown timeout (graceful shutdown exceeded its 60s hard deadline after SIGINT/SIGTERM).

Development Guidelines

  • File organization: internal/ subdirectories, Go conventions. Tests: *_test.go; E2E in internal/server/e2e_test.go. E2E prereqs: Node.js, npm, jq, a built mcpproxy binary.
  • Error handling: structured logging (zap), context wrapping, graceful degradation.
  • Config changes: update both storage and file system; the file watcher hot-reloads.
  • macOS tray dev (build / replace / verify with mcpproxy-ui-test): docs/development/macos-tray.md.
  • Windows installer: docs/github-actions-windows-wix-research.md. Prerelease (next branch + v*-rc.* tags, opt-in, off stable channels): docs/prerelease-builds.md.

Recent Changes

  • 118-pin-visibility-boundary (#1686): Go 1.26, backend-only — the operator-set profile pin is a visibility boundary on every REST/SSE read door. A pinned agent token's AuthContext.AllowedServers is bound to grant ∩ pin reach at the two seams that mint it (handleAgentTokenAuth, the SSE refresher; internal/httpapi/pin_boundary.go → auth.AuthContext.BoundToPinReach, fail-closed empty non-nil set for a deleted/empty/non-overlapping pin), so canSeeServer / visibleServers / scopeAllowedServers / every recomputed count and management.Doctor answer from one set. Profile doors: a pinned caller may learn only its own pin (profileReachable; ?profile=, /profiles/{name}, /effective-tools → the same non-disclosing 404; /profiles/active withheld for every non-admin unless selectable — the one FR-004 compatibility change). SSE: a withheld servers.changed identity takes its sibling change counts with it (every scoped caller), a pinned caller's frame is dropped only when nothing visible survives (serversChangedDeliverable, FR-007), and active_profile.changed is rendered as /profiles/active answers (renderActiveProfileChangedForCaller). Docs: docs/features/agent-tokens.md "Profile Pinning", docs/api/rest-api.md events table. No new dependencies.
  • 122-scanner-skillspector-checks: Go 1.26, backend-only — SkillSpector-informed detect checks (Apache-2.0 attribution in THIRD_PARTY_NOTICES.md): refusal.directive (anti-refusal / safety nullification / risk concealment, review), padding.hidden (horizontal ≥64, vertical ≥12 lines, block >1 KiB, repeated char ≥256, repeated line ≥16, content must follow; also "analysis incomplete" on inspection-budget exhaustion), exfil.composite (credential access + transmission + external collector within 400 bytes of one field, authentication excluded, review), payload.reconstructed (bounded declared-marker removal: quoted/tag/bare/passive/empty-replacement forms, action-governed payload ≤700 B, ≤64 removals, ≤8 active directives, forwarded shell match is HARD, limits → "analysis incomplete"), and a direct-prohibition cap in phrase.injection (detect.DirectProhibition: adverb-only binding, same raw line via detect.LineMap, no exception/reversal cue). Shared detect.FieldStrings walks description + every schema string with JSON-pointer paths and exact offsets (FieldSpan). Eval: --review-gate over detect_review_corpus_v1.json; hard gate gains the reconstruction category (self-check corpus copied). Docs: docs/features/tool-scanner.md "SkillSpector-informed checks". No new dependencies.
  • 120-scan-structural-gate: Go 1.26, Vue 3 Web UI — a deterministic structural change gate (internal/runtime/tool_contract_delta.go, contractDelta(approved, live) []Signal) runs in front of the spec-119 scan-mode approval at both checkToolApprovals gate sites: any effective safety-hint change (GATE-HINT-CHANGE, safer flips also GATE-HINT-SAFER-FLIP), a recursive input-schema widening (GATE-SCHEMA-SHAPE:<reason>:<path>: new required, type, enum added/dropped, pattern/format, loosened bound, additionalProperties relaxed, default/const, new steering property, combinators, unknown approved schema), an output-schema change (GATE-OUTPUT-SCHEMA) and a hash-equal dispatch-tier change (DeriveCallWith moved, FR-009: closes the hash-match/formula/content shortcuts, previousHintsProveRevert requires the variant too) hold the change whatever the verdict; signals go first under the MaxToolHeldSignals cap. tool_auto_approved for a changed contract carries the before-state (emitToolAutoApprovedChange). One-time startup WARN (warnScanDefaultOnce, marker scan_default_notice_done) and config.ScanDefaultTrustFinding name servers with no trust_mode. Terminology family hold_signal (contracts.AllHoldSignals, HoldSignal* TS consts, HOLD_SIGNAL_LABELS), shared fixture internal/runtime/testdata/contract_delta_cases.json (Go exact signals + vitest review-line parity), qapilot Contract.Shape/BenignWide/QP-20/S28/S29. Docs: docs/features/security-quarantine.md "Structural changes are held". No new dependencies.
  • 101-tpa-db US3 + 087-tpa-daily-refresh (tpa-bundle-refresh): Go 1.26, Vue 3 Web UI — a refresh cycle (internal/security/scanner/tpa_refresh.go: file leg then optional fetch leg, jittered tpa_refresh_interval default 24h, single-flight joins, Nudge on config.reloaded, joined in Runtime.Close) feeds the same source-neutral evaluateCandidate pipeline; the opt-in fetch (tpa_fetch_enabled default off, tpa_fetch_url no compiled-in default, env MCPPROXY_TPA_FETCH_ENABLED; Releases-API latest or direct <url> + <url>.sig, signed-only via evalEnv.MustSign, https/loopback-only transport with dial-time egress checks, host-only logs, outcomes kept apart from last_rejection) works today only with operator-signed bundles (publisher key set still empty); the activation self-check (tpa_selfcheck.go, parity-tested port of the cmd/scan-eval gate, embedded bundled/selfcheck-corpus.json) refuses a candidate as gate_failure; fetched bundles survive reload/restart (externalConfigured) and disabling fetch keeps them serving; activation observers drive the cached-only re-protection sweep (internal/server/scan_bundle_reprotect.go, held-evidence leg + Service.StartScanFromCachedContracts gated by auto_baseline_scan, marker storage.BundleReprotectState, RT-02: never approves, unquarantines or revokes); BundleInfo gains last_checked_at, last_activated_at, stale, refresh, reprotect; surfaces POST /api/v1/security/bundle/refresh, mcpproxy security bundle refresh, BundleDoctorWarnings, telemetry schema 16 refresh counters. Docs: docs/features/tpa-bundle-signing.md "Daily refresh and optional fetch", docs/features/security-quarantine.md "Re-protection after a bundle activates". No new dependencies.
  • 101-tpa-db US1 (tpa-db-format): Go 1.26, Vue 3 Web UI — external TPA bundles are signed artifacts: <bundle>.sig four-key Ed25519 sidecar (internal/security/bundlesig, pure, contract specs/101-tpa-db/contracts/sidecar-format.md), verify-before-parse source-neutral Candidate pipeline (scanner/tpa_candidate.go, FR-008 order, ceilings 8 MiB / 2000 rules / 4 KiB / 5 s via candidate_open_{unix,windows}.go no-follow descriptor opens), per-authority (publisher/operator) epoch-scoped watermark + ratchet + pin/deny-list in BBolt tpa_bundle_state (one tx with the history row) and content-addressed LKG files <datadir>/tpa-lkg/, full-digest rollback, coverageOK + async scan-mode admission degrade on the four FR-007/009a/009b conditions. Config: require_signed_bundle (default off), tpa_active_root, tpa_operator_signers, tpa_operator_baseline, tpa_operator_epoch. Surfaces: GET/POST /api/v1/security/bundle{,/history,/rollback,/unpin,/reload}, mcpproxy security bundle …, cmd/tpa-db keygen|sign|verify, telemetry schema 15 bundle keys. Publisher key set ships EMPTY. Docs: docs/features/tpa-bundle-signing.md. No new dependencies.
  • 118-auto-rescan-on-change: Go 1.26, small Web UI change — after a tool definition changes (or a tool is added after the baseline) in manual/scan trust mode the in-process TPA scan of its current definition is recorded as hold evidence (held_reason: awaiting_review + held_verdict: clean for a clean verdict; findings/coverage as before) and a debounced informational server rescan (scan_informational.go, fed by Runtime.AddToolDefinitionsChangedObserver) brings the review scan line back to "current". Switch: security.auto_rescan_on_change (default on, env MCPPROXY_AUTO_RESCAN_ON_CHANGE) AND auto_baseline_scan. RT-02 untouched: a clean verdict never approves a change. Docs: docs/features/security-quarantine.md "Automatic informational baseline scan". No new dependencies.
  • quarantine-approval-pilot: Go 1.26, tests + CI only — independent policy model internal/qapilot (no production imports), deterministic scenario inventory S01–S26 + 896-cell dispatch table + REST authz table + counting-upstream dispatch check, check ledger (QAPILOT_REPORT_DIR) gated by go run ./cmd/qapilot report, versioned mutation set tests/qapilot/mutations.json run by qapilot mutate in disposable worktrees; workflow .github/workflows/quarantine-pilot.yml (3 PR checks + scheduled rapid exploration). Docs: docs/development/quarantine-approval-pilot.md. No new dependencies.
  • 117-review-bound-approvals PR-3 (#1565): Go 1.26, backend-only — every upstream tools/list carries a managed.InventoryCapture{Seq, Epoch, Client} (process-global sequence stamped by the flight leader; ListToolsCaptured/ListToolsFreshCaptured; Client.Retire/CaptureIsCurrent; the manager retires replaced/removed clients; Manager.DiscoverToolsReportCaptured gives the sweep one capture per server). The runtime admits each inventory under the server's approval lock (admitInventoryHeld, per-server applied/definitions in toolApprovalServerLock): an older, retired or reconnected capture writes nothing (records, index, last-good snapshot + its capture, StateView); a lenient empty result is an observation fence; the approval reindex/sweep fallback re-apply the snapshot only while its capture is current, else re-list/skip. Docs: docs/features/security-quarantine.md "Tool lists are applied in the order they were captured". No new dependencies.
  • 116-config-commit-consistency PR2: Go 1.26 + a small Web UI change — removal purges the whole name-keyed purge set in the commit's bbolt transaction (internal/storage/purge.go, exact OAuth key match), in-memory purge, incarnation-bound approval writers (internal/runtime/approval_incarnation.go, REST expected_incarnation → 409 server_replaced), incarnation-conditional scan/identity/history writes. No new dependencies.
  • 117-review-bound-approvals PR-2 (#1566): Go 1.26, Vue 3 Web UI — review payload current_hash (review fingerprint rv1-… over server, tool, definition and the four safety hints; the rug-pull hash is unchanged), expected_hashes on security/approve and tools/approve (409 review_out_of_date, nothing written), server approval as one locked step (validate → baseline marker → integrity baseline + blocks → promote the reviewed pending tools → storage/config unquarantine via Runtime.Commit + UnderApprovalLock), toolreview.StaleToolReviewError, Web/CLI (review approve --from-review) binding. Both bindings coexist: a server/tool approval carries expected_incarnation (server identity, 409 server_replaced) AND expected_hashes (reviewed definitions, 409 review_out_of_date); Runtime.ApproveToolsReviewedAt / ApproveAllToolsReviewedAt / CommitServerApprovalAt check the incarnation first, under the approval lock. No new dependencies.
  • 117-review-bound-approvals (PR-1, #1567): Go 1.26, backend + copy — durable per-server tool baseline marker (bbolt bucket tool_baseline_decisions, value type in the new leaf package internal/toolreview); serverHasBaseline = marker OR approved/changed record; trusted discovery, every security approval (Runtime.CommitServerApproval, scanner ServerApprovalCommitter) set it; removed only with the server's approval records; a marker read/list failure, or a failed back-fill write on a server with approved records, aborts the discovery pass before any write; a failed first-discovery write does not (the pass's approved records establish the baseline and the next pass marks it). Docs: docs/features/security-quarantine.md "Durable tool baseline". No new dependencies.
  • 116-config-commit-consistency (PR1): Go 1.26, backend-only — one Runtime.Commit section (internal/runtime/commit.go) for every server writer, per-server generations in configsvc (internal/commitview), SaveConfiguration persists the live config (no rebuild from storage), REST PATCH / MCP update resolved inside the commit, scripts/check-upstream-write-sites.sh in CI. Docs: docs/architecture.md "Config Commit". No new dependencies.
  • 115-mcp-credential-lifecycle: Go 1.26, Vue 3 Web UI — admin MCP tool credentials (list/get/create_client/create_token/revoke) over runtime.CredentialsService, the one issue/revoke path shared with REST /tokens and /clients (secret-shaped input screen, profile_change issue/revoke, SSE credentials.changed); MCP-issued tokens are standing FR-008a bindings (guard_bound, ActiveGuardedBinding). Docs: docs/features/mcp-credential-lifecycle.md. No new dependencies.
  • 109-ux-navigation-consistency: Go 1.26, Vue 3 Web UI, Swift macOS app — Home with one needs-attention list, informed Review queue, Clients hub, catalog-first Add, grouped navigation and URL scope filters. Cross-surface parity is mechanical: internal/contracts/terminology.go + internal/contracts/testdata/terminology.json (enums), specs/109-ux-navigation-consistency/parity-matrix.json and acceptance-index.json, internal/runtime/testdata/attention_parity_* and internal/registries/testdata/catalog_github_order.json (order replays), scripts/check-no-unquarantine-callers.sh. Docs: docs/cli/attention-command.md, docs/cli/catalog-commands.md, docs/web-ui/dashboard.md. No new dependencies.
  • 108-profiles-v3: Go 1.26 (internal/profile, internal/runtime, internal/httpapi, cmd/mcpproxy), Vue 3 Web UI, Swift macOS app — profile tool policy (max_tier, unannotated, tool rules), client and token bindings, anonymous_profile, scope attribution/filters and the access explainer on every surface. Parity and enum/label contracts: specs/108-profiles-v3/parity-matrix.json, internal/profile/testdata/contract/{enums,labels,refusals}.json. Docs: docs/features/profiles.md, docs/cli/profile-commands.md. No new dependencies.