BYOS + usage limits: status, and 3 decisions I need

[suleimansh]

Status of the BYOS thread in one place. Two things I told you earlier were wrong.

Decisions I need

1. BYOS: worker/runner level, or park? The ProviderAdapter version I pitched isn't worth building (why below). Worker level is a bigger job and a different shape than I sold you.
2. Config location, only if 1 is a go: framework config (env-paths/XDG, like the dashboard prefs) or a separate dotfile.
3. [The Framework] Background jobs #298 usage guard: start now? Independent of BYOS and unblocked.

What I got wrong

"Usage isn't readable" -> it is. claude -p --output-format stream-json already emits this free on every run. No extra endpoint, no polling, no 429s:

{"type":"rate_limit_event","rate_limit_info":{
  "status":"allowed","resetsAt":1784079000,"rateLimitType":"five_hour"}}

The binary has status = allowed|allowed_warning|rejected and rateLimitType = five_hour|seven_day|seven_day_opus|seven_day_sonnet|weekly. I only observed allowed/five_hour at runtime, the rest are strings in the binary, so that's evidence and not proof.

If allowed_warning fires like the name says, your original #298 design works off the free stream. No usedPercent though, that's all Traycer's private endpoint actually buys.

"BYOS is a new ProviderAdapter" -> it isn't. Our adapter contract is tools-in / tool-calls-out with the agent loop executing them, which is model-shaped. Claude Code is an agent that owns its own tool loop. Verified: told it to read a file, and the whole tool_use -> tool_result -> text ran inside the CLI.

So an adapter can only be generation-only, which throws away the agent and pays subscription prices for text completion. And codex and gemini have no tools-off flag at all, so that shape fits exactly one provider. These CLIs are peers of a worker, not of a model. It's the Runner seam, I named the wrong one of the two.

Detail and evidence on #496. Nothing committed.

[suleimansh]

@brillout

[brillout]

3. [The Framework] Background jobs #298 usage guard: start now? Independent of BYOS and unblocked.

Yes, that would be great 🤩 I created a dedicated ticket for it:

• Consumption limits #519

1. BYOS: worker/runner level, or park? The ProviderAdapter version I pitched isn't worth building (why below). Worker level is a bigger job and a different shape than I sold you.
2. Config location, only if 1 is a go: framework config (env-paths/XDG, like the dashboard prefs) or a separate dotfile.

I don't understand most of the "BYOS is a new ProviderAdapter" -> it isn't. paragraph, so I can't I can give an opinion here.

I suspect the text was written by AI — the big issue with AI-written text is that it's very hard to understand because it "leaks" low-level implementation details (not important for the reader), but the text should be kept high-level (what the reader cares). This is a major issue I have when I try to use AI for writing documentation (e.g. the docs for Telefunc Stream).

Another problem with AI-generated text is that it often uses cryptic and unusual terms. E.g. it took me a while to understand what "BYOS" means (I had to click through multiple GitHub issues to eventually find the one GitHub issue that defines what "BYOS" means).

So I think the best would be to reframe the question, otherwise it's very hard for me to understand and answer it.

Or we use a new prompt "make text human understandable" 😀

[brillout]

Or we use a new prompt "make text human understandable" 😀

Jokes aside, actually, a prompt like keep text high-level and understandable for someone unfamiliar with the implementation details really could help. I'd be curious whether you can manage to nudge AI to write text that is easy to understand (I failed so far).

[suleimansh]

Reframing this, since my last go was unreadable. Sorry about that.

What BYOS means: people use the Claude subscription they already pay for, instead of handing us an API key.

Most of it already works. The Framework runs your own Claude Code, which already holds your subscription. It has never needed a key. That's the same reason the usage numbers in #519 work at all.

What I proposed on top was wrong. I wanted to also slot a coding CLI into the place where our SDK plugs in a model. But Claude Code isn't a model, it's a whole agent that runs its own tools. To fit it into a model-shaped hole you'd have to switch its tools off and use it only to write text: paying subscription prices for the one thing it's worst at. Codex and Gemini can't switch theirs off at all.

So the real BYOS is: The Framework drives an agent other than Claude Code. Codex, Gemini, OpenCode. That's what Traycer's front page leads with, and I want us to have it.

It fits a hole we already left. We wrap a coding agent behind one small seam: prompt it, read the code it wrote, stop it. That seam was always meant to take a second agent, so there's nothing new to design.

And both are ready. I checked: Gemini runs headless almost exactly like Claude Code, and Codex has its own non-interactive mode with JSON events. The "we don't know how to invoke them" question from my spike is answered.

Plan: Gemini first, since it's the closest shape and proves the seam cheapest. Codex next.

That answers your (1). (2) I'll come back to when there's something to configure. (3) is done: #519 shipped today.

[suleimansh]

Correcting myself an hour later: Gemini is not a candidate at all, and Codex is proven.

Gemini is out. Google switched the CLI off for individuals on 18 June and removed the Google sign-in; Pro and Ultra subscribers included. Only company licences still work. Everyone else is pointed at their new Antigravity CLI, so that, not Gemini, is the third agent whenever we get to it.

Codex works. I ran it headless on nothing but a ChatGPT subscription, no API key anywhere, and got a clean event stream back with the answer and the token counts. So it's Codex first rather than Gemini first.

One thing that falls out of it: Codex reports tokens but never a price, and says nothing about how much of the subscription is left. So the spend cap and the consumption limits stay Claude-only for now. That's fine, they were already optional per agent, but it means a Codex run is less guarded than a Claude one, and we should say so rather than let people assume otherwise.

[brillout]

I'd say let's only support Codex, and only if it's a quick-in and doesn't slows down our development in the short-term (e.g. having to toggle the usage limits UI is already a small speed bump). The goal of the MVP isn't to be perfect, the goal is to prove that we can massively increase the autonomy of AI.

(Nice, I understood 100% of the last two replies. It's 1000x better.)

[suleimansh]

All three decisions answered: Codex only (no third agent), cost made optional (#540), and consumption limits shipped. Nothing left to decide here.