fix: pipelines died at spawn on shared-branch contention (and ~ in project paths)

[bborn]

What happened

Pipeline #5120 (design-build-verify, offerlab) stalled with every step after the root parked in blocked, one second after starting:

Failed to setup worktree: create worktree on branch pipeline/5120-collab-product-network-https-linear-app: exit status 128
fatal: 'pipeline/5120-...' is already checked out at
  /Users/bruno/Projects/rails/offerlab/.task-worktrees/5120-plan-collab-product-networkhttpslinearap

Three separate bugs, all rooted in one fact: git allows exactly one worktree per branch.

1. A finished step holds the shared branch forever

A step's worktree is deliberately kept after it completes so its work can be inspected — and it goes on holding the branch. The first downstream step of every pipeline therefore couldn't attach. Not fan-out-specific; it hit sequential chains too (same error on pipeline 4963, Jul 22 and 23).

Now: a finished holder hands the branch over via git checkout --detach — moves no files, keeps uncommitted work, leaves the branch ref where it is. The worktree stays on disk and readable. A holder that is still running keeps the branch.

2. Fan-out steps were never given the branch their own instructions name

composeInstruction has always told a parallel step to push to {{branch}}-<slug>, and told its dependent to read it back with git show origin/{{branch}}-<slug>:<file>. But Create pinned only SourceBranch, so the executor tried to attach the step to the shared branch — the one its instructions explicitly say not to push to. Siblings then collided with each other and with the root.

Now Create pins BranchName = StepBranch(shared, step) for any step with a parallel peer, and the executor cuts a worktree on that branch from the shared branch. StepBranch is the single source of truth used by both, so the branch a step gets and the branch its prompt names can't drift again. Cutting a branch from a branch is unrestricted, so no number of siblings can contend.

3. A busy branch failed the step instead of waiting

The raw git error parked the step blocked with started_at and completed_at one second apart — a step that reads as "ran and finished" on every surface, having never launched an agent. That's how a pipeline silently loses a phase. Now it's ErrBranchBusy: the task stays queued and the next tick retries it.

Bonus: ~ in project paths

The same run logged git push: fatal: cannot change to '~/Projects/rails/offerlab'. Two project rows stored the shell's shorthand. config.GetProjectDir expands it; the callers reading Project.Path directly (pipeline.projectDirFor, executor.lookupKindInstructions, completion.Complete) did not — so those projects silently lost their pre-seeded branch, their project-local .taskyou/workflows dir, and any GetProjectByPath match. Now normalized on both read and write, so no consumer has to remember.

Evidence

Each test reproduces the production failure before the fix:

--- FAIL: TestSourceBranchWorktreeReclaimsBranchFromFinishedStep
    next step could not take the shared branch: exit status 128
    fatal: 'pipeline/1-demo' is already checked out at '/private/var/.../holder'

--- FAIL: TestParallelStepsGetTheirOwnBranch
    [Review A] BranchName = "", want its own branch (siblings cannot share one branch)
    both reviews got branch ""; they would contend

The executor tests drive real git repos and real worktrees — the constraint under test is git's own, so a fake would prove nothing.

• go test ./... — all green
• go vet ./..., gofmt -l — clean
• golangci-lint run with the CI-pinned v2.8.0 — 0 issues

No user-visible surface changes (daemon/git internals), so no screenshots.

🤖 Generated with Claude Code

[bborn]

Verified live on the stuck pipeline (#5120, offerlab)

Deployed this build to the daemon and retried the two review steps that had been dead since 15:24.

Two further bugs surfaced on the first real run — both were unreachable until fan-out steps could actually spawn together, and both are fixed in 1e093a60:

1. git worktree add is not concurrency-safe within a repo. Siblings spawning in the same instant raced on .git/config, and the loser fails outright rather than retrying:

   error: could not lock config file .git/config: File exists
   error: unable to write upstream branch configuration
   

   Now serialized with a per-repo flock (executorlock.AcquireRepo). This covers the ordinary-task path too — two normal tasks starting at once raced the same way, which likely explains intermittent worktree failures unrelated to pipelines.

2. -b X <path> origin/<shared> silently set X's upstream to the shared branch. A later bare git push from a fan-out step would have landed its commits on the branch its instructions explicitly forbid. Cut with --no-track now.

The concurrency test fails without the lock and passes with it:

with lock:     ok    github.com/bborn/workflow/internal/executor
without lock:  FAIL  github.com/bborn/workflow/internal/executor

Live result

Both review steps now hold their own branches at once — which had never happened before:

.task-worktrees/5121-planreviewa-...  [pipeline/5120-...-planreviewa]   processing
.task-worktrees/5122-planreviewb-...  [pipeline/5120-...-planreviewb]   processing

Unrelated observation for later: on its first run, PlanReviewB ran for 4 minutes on kimi-k2.7-code:cloud and committed nothing, so the step parked instead of handing off. That's a step/model question, not a plumbing one — the worktree and branch were correct.