diff --git a/skills/run-workflow/SKILL.md b/skills/run-workflow/SKILL.md index c741c86..a512b92 100644 --- a/skills/run-workflow/SKILL.md +++ b/skills/run-workflow/SKILL.md @@ -44,6 +44,20 @@ sim --output json workflows run \ Do not guess a source run or synthesize upstream outputs. Confirm that the source run belongs to the workflow and contains the state the selected block needs. +## Runs longer than about a minute + +A manual run holds its HTTP connection open for the entire execution, and `--manual` refuses +`--async`. On hosted deployments the fronting load balancer drops idle connections after roughly a +minute (observed; re-verify on the current deployment), and a dropped connection cancels the run. +The failure surfaces as a transport error such as `Could not reach : fetch failed`, not as +a timeout, so it reads like network flakiness. It is not: retrying the same synchronous run fails +the same way, and a sequence of such attempts corrupts the evidence - a deterministic workflow +starts looking nondeterministic because most of its recorded attempts are transport casualties. + +When a workflow can plausibly exceed a minute, deploy it and run `--async`, then wait with +`workflows runs wait` or poll `workflows runs get` with a stopping bound. Reserve synchronous +manual runs for graphs that finish quickly. + ## Keep output focused - Use repeated `--select-output ` values when only specific outputs matter. @@ -61,6 +75,23 @@ workflow and contains the state the selected block needs. 4. Correct the graph with the build skill. Do not hide a deterministic failure behind retries or a different execution mode. +Four properties of runs and run records that mislead diagnosis when unknown: + +- A run record has a lifecycle. `logs get` returns NOT_FOUND for a run that is still in flight and + for one that was cancelled, and a completed run can report a `redacting` status for a minute or + two before its content is readable. Poll with a bounded retry before concluding the run vanished. +- Firing a run immediately after `workflows deploy` reports the new version active can still + execute the previous deployment, silently. When a run exists to verify a deploy, confirm the + executed behavior from its outputs rather than trusting the deploy response, and re-run if the + outputs match the old version. +- Run traces nest: a child workflow's blocks appear inside its parent span, not at the top level. + Reading only the top level of the span tree silently drops every block a child workflow ran, + which can be most of the run. Walk the tree recursively before counting blocks or hunting the + failing one, and judge the block-level trace rather than the wrapper status. +- Cost comes in two units. The CLI's `logs get` reports `cost.total` in dollars, while the + in-workflow logs block reports the same run's cost in credits. Never compare or store the two as + one number. + Report which mode ran, the terminal status, and the relevant output or error. Include the run id when the selected execution mode returns one; `--follow` streams omit it. Never print profile credentials or raw secrets from block inputs.