fix(executor): stale tmux window no longer pins a shared branch forever

[bborn]

The stall

Task 5128 ([Verify] step of the Collab Product Network pipeline) finished its work — committed, pushed, opened a PR — then re-queued itself every 2 seconds for 38 minutes.

Diagnosis, from the live system:

1. The Build step (5124) still held the shared branch checked out:

   worktree .task-worktrees/5124-build-collab-product-networkhttpslineara
   HEAD    aaa3cac90   ← the verify step's own commit
   branch  refs/heads/pipeline/5120-collab-product-network-https-linear-app
   

2. Task 5124 was done, but its tmux window task-5124 was still open.
3. taskIsLive consults tmuxWindowExistsForTask regardless of status → read the leftover window as "still working".
4. releaseBranchFromFinishedHolder refused to reclaim → addSourceBranchWorktree returned ErrBranchBusy.
5. executeTask re-queued the step. Worker ticker is 2 * time.Second. Repeat forever.

An executor window routinely outlives its agent: when the agent exits the pane falls back to a plain shell, and nothing reaps the window. So this is the normal end state of a finished step, not an edge case — any completed step can pin its branch this way.

Why the spin made it worse

ErrBranchBusy re-queueing rather than failing is right, but unbounded it is a hot loop. Each pass re-enters executeTask and re-logs Starting task #N. ~1,100 passes flushed the task's entire log history out of the 1000-entry ring buffer — the only record of what the step had actually done was destroyed by its own retries. It also made the task un-parkable by hand: setting it blocked was overwritten by the next tick's re-queue. Only close escaped.

The fix

1. A terminal status is authoritative. taskIsLive stops consulting tmux for done/archived. The window stays the tiebreaker for a non-terminal task, since a blocked step may still have a live agent sitting on a question.

2. Deferrals back off and give up. Contention deferrals now back off geometrically (5s → capped at 2m) and park in blocked after 30m, so a branch that never frees surfaces to a human instead of spinning silently. processNextTask gates deferred steps on that backoff — that is what stops the log spam.

Also folds the duplicated windowExistsFn override into an e.windowExists helper.

Tests

Four new cases in shared_branch_test.go, on real git repos like the existing ones:

• ...ReclaimsFromDoneHolderWithStaleTmuxWindow — the regression. Verified it fails without the fix, with the exact production error:

  a done holder's stale tmux window still pins the branch: branch is checked out by
  a running step: pipeline/1-demo is checked out at .../holder
  

• ...KeepsBranchForBlockedHolderWithLiveWindow — guards against over-correcting; a blocked step with a live window keeps its branch.
• ...ReclaimsFromBlockedHolderWithNoWindow — an abandoned blocked holder releases.
• TestBranchWaitBacksOff / TestBranchWaitGivesUpAfterDeadline — backoff grows, respects the cap, gates the retry, isolates per-task, and gives up on elapsed time rather than attempt count.

go vet ./..., go test -race ./... (full repo, incl. the parity harness), and make lint (0 issues) all pass.

Not covered here

The verify step also never signalled completion after its successful run — it sat in processing, which is what fed it into the retry loop to begin with. That is a separate bug in the pipeline completion handoff and is not addressed by this PR.

The task-5124 window is still open on the affected machine; until it is closed, that branch stays pinned under the old binary.

🤖 Generated with Claude Code