Update best practices for Dev Proxy v2.2.0#113
Conversation
waldekmastykarz
left a comment
There was a problem hiding this comment.
Let's add info about random ports which is invaluable for detached mode, especially combined with asSystemProxy false
|
|
@garrytrinder I've opened a new pull request, #114, to work on those changes. Once the pull request is ready, I'll request review from you. |
Best Practices Document Review — ReportContextThe Summary of changes1. New content for Dev Proxy v2.2.0Added rules covering new v2.2.0 features:
2. Tone: directive, not suggestiveProblem: The original document used hedging language — "consider using", "prefer specifying", "you could" — which is appropriate for human documentation but ambiguous for an LLM. An agent has no judgment to apply to "consider"; it needs a clear instruction. Change: Every rule is now a direct instruction. "Consider using the LatencyPlugin" became "When mocking APIs, use the LatencyPlugin." Where exceptions exist, they're stated explicitly: "Store files in Reasoning: LLMs follow instructions more reliably when they are unambiguous. Hedging language creates unpredictable agent behavior — sometimes following the suggestion, sometimes not, with no consistent rationale. 3. Severity: everything is mandatory with explicit exceptionsProblem: An initial pass introduced MUST/SHOULD/MAY severity levels (RFC 2119 style). However, SHOULD is ambiguous for an agent — it has no criteria to decide when to follow a recommendation and when to skip it. Change: Dropped SHOULD and MAY entirely. Every rule is mandatory (stated once in the header). Where a rule has exceptions, the exception is explicit and conditional: "unless the user explicitly requests YAML", "unless a specific port is required." Reasoning: An agent needs deterministic rules. "SHOULD do X" gives no decision criteria. "Do X, unless Y" gives a clear rule with a clear escape hatch. 4. Format: flat numbered list with domain tagsProblem: The original used markdown headings with per-section numbered lists. This created several issues for LLM consumption:
Change: Replaced headings + per-section lists with a single continuously numbered list (1-34). Each rule is tagged with a domain: Reasoning:
5. Removed fluff and redundancyProblem: The original contained explanatory prose ("Hot reload helps you iterate faster", "This allows you to define the URLs in one place and have them applied globally") and repeated rationale across sections. Motivational or explanatory text doesn't help an agent act — it just consumes tokens. Change: Stripped all explanatory prose. Each rule states what to do and, where relevant, when to do it. Nothing more. Reasoning: Every token in the context window matters. Explanatory text competes with actionable instructions for the model's attention. Removing it makes the rules denser and more likely to be followed. 6. Conditions front-loadedProblem: Version conditions were buried in parentheses: "use Change: Conditions are now at the start of rules: "When version >= 2.2.0, run Reasoning: Front-loading conditions makes them harder to skip. The agent evaluates the condition before reading the instruction, reducing the chance of applying a v2.2.0 rule to an older version. Before/after comparison
|
| 15. [urls] Define `urlsToWatch` globally in the config file. Use plugin-specific `urlToWatch` only when you need to override the global config for a specific plugin. | ||
| 16. [urls] To exclude a URL, prepend it with `!`. | ||
| 17. [urls] To match URL patterns, use the `*` wildcard. | ||
| 18. [plugins] Before adding a plugin, use the `FindDocs` tool to verify it exists and get its documentation. |
There was a problem hiding this comment.
Not sure how widely used, but this could interfere with custom plugins for which we don't have docs. Might be good to test if the agent understands the difference.
Updates the best practices document with new features and guidance from Dev Proxy v2.2.0:
.yaml/.ymlas supported config formats alongside JSONdevproxy config validate— new subcommand for pre-run config validationdevproxy config new --format yaml— creating YAML configs@dynamic=initialvalue— per-response retry-after with initial values in GenericRandomErrorPlugin--no-watchflag — disable hot reload for CI/CD and automation--output json|text— structured output for automation