Skip to content

feat(pi-ext-usage-report): @agentic-kit/pi-ext-usage-report — self-report a pi session's usage to /v1/usage - #1726

Merged
pyramation merged 4 commits into
mainfrom
feat/pi-ext-usage-report
Aug 14, 2026
Merged

feat(pi-ext-usage-report): @agentic-kit/pi-ext-usage-report — self-report a pi session's usage to /v1/usage#1726
pyramation merged 4 commits into
mainfrom
feat/pi-ext-usage-report

Conversation

@pyramation

Copy link
Copy Markdown
Contributor

Summary

Package 4 of the additive @agentic-kit/* series (planning issue #1657). The local / own-provider-key metering lane: a pi extension that turns each assistant message's usage into a POST /v1/usage on the existing gateway, so a run whose model calls don't pass through agentic-server still lands in inference_log. No gateway change — that endpoint already exists.

The complement of @agentic-kit/pi-ext-metered-model (#1725), which is authoritative because the gateway does the counting; this one is self-reported, so it is for usage visibility and reconciliation, not tamper-proof billing. The README states that boundary.

Three decisions worth reading the code for:

  • input_tokens includes cache tokens. pi reports {input, output, cacheRead, cacheWrite, totalTokens, cost}; input excludes cache hits, so reporting it alone under-reports a long agent session by an order of magnitude:

    input_tokens = input + cacheRead + cacheWrite   // split + pi's cost breakdown survive in raw_usage

    A turn that never reached the provider (no usage) reports nothing rather than a row of zeros; a failed turn is reported (status: 'error', provider message as error_type) because those tokens were spent.

  • Dedupe on responseId. pi can emit message_end twice for one response (rewritten message, replay on resume), which would bill twice; messages without a responseId are reported as-is.

  • Delivery is queued, not awaited in the handler — an HTTP round trip per assistant message would sit directly in the agent's turn latency. Failures are not lost: the first is retained and rethrown from flush(), which also runs on session_shutdown, so a host that shuts down cleanly fails loudly instead of silently dropping usage. onError opts into non-fatal delivery.

const usage = createUsageReportExtension({
  gatewayUrl: 'https://agentic.example.com',       // gateway root, not /v1
  identity: { databaseId, entityId, actorId, runToken }
});
// hand usage.extension to pi; usage.flush() at shutdown

sink replaces HTTP entirely (in-process metering, a queue, a test double). Identity headers and gateway-URL validation are reused from @agentic-kit/pi-ext-metered-model rather than re-derived, so both lanes send the same X-Database-Id/X-Entity-Id/X-Actor-Id/bearer contract and both reject a gatewayUrl ending in /v1 at construction.

Tests: 28 across report shaping, the HTTP sink (injected fetch), and queue/failure semantics — no gateway or credentials needed. Registered in the agentic CI shard.

This branch is stacked on #1723#1724#1725, so its diff against main carries those packages until they merge.

Link to Devin session: https://app.devin.ai/sessions/450ce6d6659c47759c184ae6ec19a2a8
Requested by: @pyramation

A pi extension that drains pi's append-only session into an
@agentic-kit/run-log store — verbatim, in order, same code locally and in
the cloud (only runId and the store differ).

Draining is index-based because the session is append-only; the read
position is keyed to the session header id so a switch/fork re-mirrors and
the store's idempotency absorbs the duplicates. A drain advances only after
a successful append, and store failures are rethrown into pi's dispatch by
default rather than silently dropping entries.
@pyramation pyramation self-assigned this Aug 14, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@pyramation
pyramation merged commit a71cae2 into main Aug 14, 2026
36 of 37 checks passed
@pyramation
pyramation deleted the feat/pi-ext-usage-report branch August 14, 2026 05:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant