Detect Claude auth expiry before it silently stops every agent

[bborn]

The problem

TaskYou-OS agent servers authenticate Claude Code per-Unix-user via claude /login (claude.ai OAuth against a Max subscription). That login expires roughly every 30 days. When it lapses, every agent task fails or stalls — and there was no detection at all. The operator found out by noticing the board had gone quiet.

Why the obvious check fails

claude auth status cannot be trusted for this. It reads cached local config and never contacts Anthropic, so it reports success on credentials that have been dead for months. On a 95-day-dead login it still returned, with exit 0:

{
  "loggedIn": true,
  "authMethod": "claude.ai",
  "apiProvider": "firstParty",
  "subscriptionType": "max"
}

claude doctor, claude mcp list and claude auth status --text are equally offline and equally wrong. Any monitor built on them reports a healthy server right up until someone asks why nothing has run in a week.

The detection method

1. Free offline pre-check. Read ~/.claude/.credentials.json → claudeAiOauth.refreshTokenExpiresAt (epoch ms). The access token lasts ~8h and self-refreshes; the refresh token is the real clock (~30 days, sliding). Three outcomes:

Credentials file	Verdict
refreshTokenExpiresAt in the future	ok + days remaining → gate the probe
refreshTokenExpiresAt in the past	expired → skip the probe, spend no tokens
claudeAiOauth present, no refreshTokenExpiresAt	pre-2.1.x format, certainly dead → skip the probe
No claudeAiOauth block at all, or no file	unknown, not dead → let the probe decide

That last row matters: on macOS the credential lives in the Keychain, so the file has only an mcpOAuth block yet auth works fine. Treating "no block" as "dead" would false-alarm every such box. Verified against a real machine in that state.

2. The only truthful probe — one real model request:

claude -p "hi" --model haiku --max-turns 1 </dev/null >/dev/null 2>&1

Exit 0 = healthy (measured 3–6s). Exit 1 on expired auth with "Failed to authenticate: OAuth session expired and could not be refreshed".

Token cost

~40 tokens per probe. At the default 30-minute cron that is ~48 probes/day ≈ 2k tokens/day per server — and less in practice, because the free offline check short-circuits the probe entirely once a credential is provably dead. Nothing is spent proving something already known.

What ships

A. templates/claude-auth-monitor.tmpl (new)

Free check → gated probe → act:

• Healthy — remove .auth-failed; if the refresh token expires within 5 days, append an auth_expiring event carrying days_remaining.
• Unhealthy — touch .auth-failed, append an auth_failed event.

Idempotent and cron-safe. Alerts are rate-limited (failures re-alert every 6h, warnings every 24h) so a 30-minute cron cannot storm Slack. A missing claude binary logs and exits 0 rather than raising a false auth alarm.

This finally activates dead code. modules/linear/linear-poll.mjs has always read a .auth-failed flag and degraded gracefully (create tasks, don't execute them) — but nothing in the repo ever wrote it. The monitor writes it at exactly the path the poller resolves: join(__dirname, '.auth-failed'), i.e. ~/scripts/.auth-failed.

B. Cron wiring in setup.sh

New install_auth_monitor(), called from both the server and exe.dev provisioning paths, installing a */30 cron entry so new installs are covered automatically.

Multi-tenancy: every path the monitor writes — flag, state, log — lives under the invoking user's own $HOME, never a shared /tmp. Several GMs can share one box, and a /tmp/ty-daemon.log owned by another user is exactly how a daemon start got broken in production. The script also falls back to $HOME at runtime if the rendered home is not writable, and the local staging file is PID-unique.

C. Slack bridge cases

formatNotification gained auth_failed and auth_expiring. These carry no task_id, so they route to SLACK_NOTIFY_CHANNEL rather than a task thread — previously they would have rendered as Task #? (auth_failed). Handles singular/plural days and a missing day count without printing NaN.

D. /doctor Check 9 — and what it rectifies

Reports days until refresh-token expiry, runs the probe once, and then fixes existing installs rather than just reporting on them:

1. Installs the monitor + cron on GMs that predate this PR. Every server provisioned before now has zero expiry detection; doctor renders the template, deploys it, and adds the cron entry.
2. Clears a stale .auth-failed flag — if the flag is present but the probe returns AUTH_OK, the login was fixed but the flag was never cleared, and linear-poll.mjs is still refusing to execute tasks. The monitor would clear this itself within 30 minutes; doctor does it immediately.
3. Warns at ≤5 days remaining with the exact claude /login fix.

E. .claude/commands/launch.md

Stopped treating loggedIn: true as proof that auth works. The credential-transfer step now verifies with the probe, and the section leads with why claude auth status must not be used.

Docs

README.md gains a Claude Auth Expiry section: the ~30-day cycle, the lying output above, the detection method, and claude setup-token — which issues a 1-year token but disables claude.ai MCP connectors and Remote Control, so it is documented as an option for boxes that don't need those, explicitly not the default.

Verification

Every piece was tested locally against a throwaway $HOME. No real ~/.claude/.credentials.json was modified, and the live probe was run exactly twice (once to confirm exit code and timing, once via the existing QA path).

Monitor script — rendered through the same substitution setup.sh uses, then driven through 9 scenarios with a stub claude binary that counts invocations:

Case	Expected	Result
Healthy creds (30d), probe OK	no flag, no event, 1 probe	pass
Creds expiring in 3d, probe OK	auth_expiring w/ days_remaining, no flag	pass
Immediate re-run	second alert suppressed by cooldown	pass
refreshTokenExpiresAt in the past	flag set, auth_failed, 0 probes	pass
Pre-2.1.x creds (no field)	flag set, auth_failed, 0 probes	pass
Healthy creds but probe fails	flag set, auth_failed	pass
Re-run while failed	second alert suppressed	pass
Recovery (probe OK again)	flag removed	pass
No claudeAiOauth block (Keychain)	not treated as dead — probe runs, no flag	pass
Credentials file missing	probe runs, no flag	pass
claude binary absent	exit 0, no flag, no false alarm	pass
SERVER_HOME unwritable	falls back to $HOME	pass

Rendered output has zero unsubstituted {{...}} placeholders and passes bash -n.

Slack bridge — node --test in modules/slack: 14/14 pass, including new coverage for both event types. Additionally piped the monitor's literal emitted bytes through the bridge's real readNewChunk → formatNotification path and confirmed both render correctly and route to the notify channel (no task_id).

setup.sh — bash -n clean. install_auth_monitor() was extracted and executed verbatim against stubbed ssh/scp: correct mkdir/scp/chmod sequence, cron entry installed once, idempotent on re-run, and no state path under /tmp.

/doctor snippet — the embedded days-remaining Python is quoted to survive an ssh '...' wrapper; executed through that exact quoting against all five credential shapes, returning DAYS_LEFT=N, PRE_2_1_X_FORMAT_DEAD, NO_OAUTH_BLOCK, and NO_CREDENTIALS_FILE correctly.

Repo QA harness — qa/run-qa.sh: 29 passed, 0 failed.

Rebased on main at 0def18b (post-#41).

🤖 Generated with Claude Code

https://claude.ai/code/session_019jCVFwrwj2ajbbcnBWpR8z