Skip to content

🤖 feat: add provisional default-hidden GPT-6 Sol model entry - #4259

Draft
ThomasK33 wants to merge 6 commits into
mainfrom
gpt-6-sol-prep
Draft

ThomasK33 wants to merge 6 commits into
mainfrom
gpt-6-sol-prep

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Sep 15, 2026

Copy link
Copy Markdown
Member

Important

DRAFT — do not merge. gpt-6-sol is NOT an officially announced OpenAI model. This PR is deliberate advance preparation. Its pricing is an explicitly PROVISIONAL ESTIMATE (maintainer-authorized), not official data; see the before-undraft checklist.

Summary

Adds openai:gpt-6-sol to the curated known-model registry as a provisional, default-hidden entry so Xum can enable it quickly if OpenAI officially releases it. The model is hidden in the selector, has no aliases, is not warmed, gets the conservative default thinking policy, and is excluded from automatic compaction suggestions. Its stats (limits + pricing) are a maintainer-authorized provisional estimate copied 1:1 from the repo's effective GPT-5.6 Sol baseline.

Background

Lead: an X post claiming Opus 5.1 and GPT-6 Sol are imminent (not official), plus community reports that the id gpt-6-sol appeared on the OpenAI API (Reddit r/singularity, HN 49665088). Verification, calibrated to the sources checked (OpenAI announcement pages, developer-docs search results, news coverage) as of 2026-09-15:

Implementation

Follows the default-hidden pattern from the Daybreak models (#4113) rather than the first-class pattern used for Astra (#4064), because Astra's PR encoded confirmed capabilities and none exist here.

  • knownModels.ts: GPT_6_SOL entry — no aliases (sol stays on GPT-5.6 Sol), not warmed, openai/gpt-5 tokenizer approximation, ordered after GPT-5.6 Sol/Astra; added to DEFAULT_HIDDEN_MODELS.
  • config/index.ts + appConfigOnDisk.ts: new seed-once migration flag gpt6SolModelHidden; the Daybreak seed block is generalized into per-flag seeds (flags are frozen history — a flag that already ran never re-hides its models). The seed preserves the pending local-preference migration marker (hiddenModelsInitialized flips only on the first-ever seed pass) and treats explicit config references (default model, advisor model, agent AI defaults, fallback chains) as prior opt-ins that are never hidden.
  • Provisional pricing estimate (models-extra.ts): pricing and token limits copied by reference from GPT_56_SOL_STATS — the repo's effective gpt-5.6-sol baseline ($5/M in, $30/M out, 1.05M context, 272K long-context tier at 2x in / 1.5x out, cache costs). The models-extra override wins over LiteLLM's stale models.json row ($4/$20, 922K) at runtime, so the effective baseline is unambiguous; the discrepancy is upstream staleness, already documented in the baseline's comment. Capability flags and knowledge cutoff are deliberately omitted (nothing official is published). A registry test pins estimate ↔ baseline equality so divergence must be a conscious decision.
  • Budget semantics under the estimate (accepted trade-off, maintainer-authorized): goal budgets accept the model (nonzero rates) and CLI --budget enforces the cap against estimated rates. If real pricing lands higher (Astra bills 2x Sol), spend is under-counted until the entry is corrected — hence the required checklist item below. This supersedes the interim null-stats fail-closed approach from the review loop.
  • compaction/suggestion.ts: automatic higher-context suggestions skip every DEFAULT_HIDDEN_MODELS entry (also closes the same pre-existing exposure for the Daybreak tiers); explicit user-configured compaction models are unaffected.
  • Thinking policy: no changesgpt-6-sol falls through to the default ["off","low","medium","high"] policy; tests pin that it does not inherit Astra's or the GPT-5.6 family's native-max/no-off surfaces.
  • Docs: regenerated models table plus a note marking the model hidden-by-default with explicitly provisional, possibly-understated pricing.

Before-undraft checklist (REQUIRED)

Do not mark this PR ready for review/merge until every item is done against official OpenAI documentation:

  • Replace the provisional pricing estimate in models-extra.ts with the official model-page rates (incl. cached-input, cache-write, long-context tier and threshold) and update the baseline-equality pin in knownModels.test.ts accordingly.
  • Verify the official model id is exactly gpt-6-sol (currently based on community API sightings only); rename/remove the entry if it differs.
  • Verify official capabilities: context window / max output, reasoning surface (native max? rejects none? compare Astra's openaiRejectsDisabledReasoning), reasoning.mode: "pro", tool calling on Chat Completions vs Responses-only, vision/PDF, knowledge cutoff, tokenizer (currently approximated with openai/gpt-5).
  • Verify availability: the model is actually servable via the API (tiers/rollout), before exposing it by default.
  • Decide alias policy (sol/gpt staying on GPT-5.6 Sol vs moving) — deliberate product decision, not part of this PR.
  • Decide whether to remove it from DEFAULT_HIDDEN_MODELS for new installs (existing users keep their visibility choices; the seed flag never re-runs).
  • Decide whether to keep it excluded from automatic compaction suggestions once released.
  • Add a usage-to-budget-stop test against the real pricing (the current test pins only estimate-baseline equality).
  • Compact display collision: both GPT-5.6 Sol and GPT-6 Sol compact to "Sol" in the composer (pre-existing tier-regex behavior); disambiguate if both stay visible.
  • If LiteLLM publishes the model upstream, reconcile with the models-extra entry per that file's convention.

Validation

  • make static-check green on every pushed head (lint, typecheck, fmt, docs sync, generated content).
  • Focused suites green on final head: node config migrations (frozen-history, migration-marker, opt-in seed tests), known models (incl. estimate↔baseline pin), model catalog, update-models coverage, model stats, thinking policy/types, compaction suggestion (red-green probe of the hidden-model exclusion), useModelsFromSettings, WorkspaceContext (+ model migration), providerService, ACP config options, IPC model preferences.
  • Full local unit suite: passes except known host-environment flakes in files untouched by this diff (git-template fixtures, three load-sensitive UI/timer suites); all pass in isolation.
  • The prior Test / Unit CI failure (listModelCatalogIds required every curated id to resolve stats) is fixed by the stats entry itself — no checks were weakened: the interim coverage/catalog exemptions were reverted and both checks are back to full strictness.

Risks

The model is default-hidden and alias-free, so no default flow changes. Cost tracking/budget enforcement for this specific model relies on an estimate that may understate real spend (documented above and in code; accepted trade-off per maintainer authorization). The hidden-model seed migration is covered by the pre-existing suite plus new frozen-history/marker/opt-in tests.

Review record (cap exhausted — no further cycles)

Instruction budget: six completed code/security reviews plus one independent advisor pass — now exhausted. No further review or advisor cycles will be requested on this draft.

Round Head Reviews Findings Outcome
1 01ca053 Codex code + security (auto-paired) P2: hidden models auto-suggested by compaction; P2: explicit $0 pricing bypasses CLI --budget Fixed in 252bba9, cb5ac4b; threads replied + resolved
2 cb5ac4b Codex code + security (auto-paired) P2: later seed flips hiddenModelsInitialized (legacy local hides droppable); P2: seed hides models the config already references Fixed in 64169a3; threads replied + resolved
3 64169a3 Codex code + security (auto-paired) none Clean on both loops
Final Independent advisor (fresh context) recommended scope reduction + accuracy fixes Accuracy fixes applied (f04892f); scope decision stays open at release

Unreviewed post-cap commits (maintainer-authorized): f04892f (comment accuracy only) and 782cca6 (provisional pricing estimate replacing the null-stats approach). 782cca6 intentionally supersedes part of the round-1 security fix: the security review recommended treating unpublished pricing as unknown or rejecting the model under budgets; the maintainer explicitly authorized an estimate instead. This finding is therefore resolved by authorized trade-off, not by code, and remains listed below until official pricing lands. Codex verdicts are bound to 64169a3.

Unresolved assumptions / open items

  • gpt-6-sol existence, specs, pricing, capabilities, and availability are entirely unconfirmed (see checklist).
  • Budget enforcement for this model uses estimated rates and may under-count until official pricing replaces the estimate (authorized trade-off; supersedes the round-1 security fix's fail-closed behavior).
  • Provider-catalog opt-ins (separate providers store) and per-workspace AI settings are not consulted by the config-load seed; affected early adopters re-enable once in Settings → Models.
  • Advisor's scope-reduction alternative (drop the entry + migration until official release) remains open for the maintainer.
  • This draft is not claimed ready: post-cap commits are unreviewed by the loop, and the before-undraft checklist is unmet by design until OpenAI publishes official data.

Generated with xum • Model: anthropic:claude-fable-5 • Thinking: xhigh

Prepare openai:gpt-6-sol (unannounced; community API sightings only) as a
default-hidden known model so it can be enabled quickly at release. No aliases,
no warm, no speculative thinking policy or pricing. Adds a gpt6SolModelHidden
seed-once migration flag so existing configs hide it without re-hiding models
users re-enabled after the Daybreak seed.
@mintlify

mintlify Bot commented Sep 15, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
Mux 🟢 Ready View Preview Sep 15, 2026, 12:45 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

This comment has been minimized.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 01ca0537be

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/common/constants/knownModels.ts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛡️ Codex Security Review · Automatically triggered

Here are some automated security review suggestions for this pull request.

Reviewed commit: 01ca0537be

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Comment thread src/common/utils/tokens/models-extra.ts Outdated
…tions

Codex round-1 finding: getHigherContextCompactionSuggestion scanned every
KNOWN_MODELS entry, so restricted/provisional default-hidden models (Daybreak,
GPT-6 Sol) could be offered as Compact & retry targets. Skip DEFAULT_HIDDEN_MODELS
in the automatic scan; explicit user-configured compaction models are unaffected.
Codex security round-1 finding: explicit $0 stats made getTotalCost return a
defined $0, bypassing the CLI --budget unknown-pricing rejection and the
goal-budget modelHasPricingData guard on a billable API model. Remove the
provisional models-extra entry (null stats = unknown pricing = fail-closed
budgets) and exempt provisional entries from the curated-coverage bar.
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: cb5ac4b075

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cb5ac4b075

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/config/index.ts Outdated
Comment thread src/node/config/index.ts Outdated
…n hidden-model seeding

Codex round-2 findings: (1) a later seed flipped hiddenModelsInitialized just
because an earlier seed had created the array, letting the browser migration
drop legacy local hides — flip only on the first-ever seed pass; (2) seeding
hid models the config already references (defaultModel, agent AI defaults,
fallback chains) — explicit references now count as opt-ins and are skipped.
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 64169a3fa5

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33

Copy link
Copy Markdown
Member Author

Review-gate bookkeeping: the Codex Review Summary card (node IC_kwDOPxxmWM8AAAABUnaNuQ, body sha256[0:16] d1158341d515a122) was minimized as RESOLVED after verifying: all 4 findings across rounds 1–2 are fixed with inline replies, all review threads are resolved, and both the Code Review and Security Review rows show Completed on the current head 64169a3 with no new findings. The Codex Comments CI job counts the mutable summary card as unresolved even after settlement (known checker limitation); minimizing only that card is its supported resolution. This will be re-assessed if the head or review state changes.

…zero rates)

Advisor accuracy pass: modelHasPricingData already treats all-zero rates as
unpriced, so only the CLI --budget unknown-pricing rejection was bypassed by a
defined $0 total. Comment-only change.
…PT-5.6 Sol baseline

Maintainer-authorized change: replace the null-stats fail-closed approach with
an explicitly labeled PROVISIONAL ESTIMATE copied 1:1 (by reference) from
GPT_56_SOL_STATS, the repo's effective gpt-5.6-sol baseline. Budgets now
enforce against estimated rates instead of rejecting the model; if real
pricing lands higher, spend is under-counted until the entry is corrected
(required before-undraft checklist item). Reverts the now-unneeded
curated-coverage and Treat-as catalog exemptions so those checks return to
full strictness, and pins estimate-baseline equality in the registry test.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant