fix: stale modelSlug #99

4ndrelim · 2026-01-24T22:29:45Z

When users input BYOK in the middle of a conversation and resume conversation with the same model, backend uses a stale version of the modelSlug which may cause invalid-model-id error as OpenRouter expects OpenAI models to be prefixed with openai/ whereas OpenAI API does not require so.

Key Change is in setting-text-input.tsx. Every time settings is updated, we force an update to modelSlug.

We attempt to do exact suffix matching (and do not assume provider prefix) for modelSlug. e.g. gpt-4o will be matched to openai/gpt-4o and vice-versa but will not match openai/gpt-4o-mini. The logic is in setting-text-input.tsx
Note some of the unresolved below. @wjiayis / myself have provided suggested fixes but there might be room for improvement.

For more info, see the issue documented: #98

Copilot

Pull request overview

This pull request fixes a stale modelSlug issue that occurs when users provide their own API key (BYOK - Bring Your Own Key) in the middle of a conversation. The core problem is that OpenRouter expects OpenAI models to be prefixed with openai/ (e.g., openai/gpt-4.1), while the standard OpenAI API does not use this prefix (e.g., gpt-4.1). When users switch between these modes, the backend was using a stale model slug that could cause invalid-model-id errors.

Changes:

Frontend automatically refreshes the model list and updates the current conversation's modelSlug when the OpenAI API key is changed
Backend health check function now accepts configurable base URL and model parameters to properly validate both OpenRouter and OpenAI endpoints
Backend endpoint selection logic updated to use the standard OpenAI base URL when users provide their own API key

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File	Description
webapp/_webapp/src/views/settings/setting-text-input.tsx	Adds logic to fetch updated model list and update conversation modelSlug when openaiApiKey setting is changed
internal/services/toolkit/client/utils_v2.go	Updates CheckOpenAIWorksV2 signature to accept baseUrl and model parameters for flexible health checking
internal/services/toolkit/client/client_v2.go	Modifies GetOpenAIClient to use OpenAIBaseURL for BYOK scenarios and updates NewAIClientV2 to perform conditional health checks

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

internal/services/toolkit/client/client_v2.go

Copilot · 2026-01-24T22:33:54Z

internal/services/toolkit/client/client_v2.go

+		CheckOpenAIWorksV2(oaiClient, cfg.InferenceBaseURL+"/openrouter", "openai/gpt-5-nano", logger)
+	} else {
+		oaiClient := openai.NewClient(
+			option.WithBaseURL(cfg.OpenAIBaseURL),
+			option.WithAPIKey(cfg.OpenAIAPIKey),
+		)
+		CheckOpenAIWorksV2(oaiClient, cfg.OpenAIBaseURL, "gpt-5-nano", logger)


The model used for the health check ('gpt-5-nano') may not exist or be available with user-provided API keys. When using BYOK (Bring Your Own Key), the health check should use a model that is guaranteed to exist in the standard OpenAI API, such as 'gpt-4o-mini' or 'gpt-4o'. Using 'gpt-5-nano' could cause the health check to fail even when the API key is valid.

Suggested change

CheckOpenAIWorksV2(oaiClient, cfg.InferenceBaseURL+"/openrouter", "openai/gpt-5-nano", logger)

} else {

oaiClient := openai.NewClient(

option.WithBaseURL(cfg.OpenAIBaseURL),

option.WithAPIKey(cfg.OpenAIAPIKey),

)

CheckOpenAIWorksV2(oaiClient, cfg.OpenAIBaseURL, "gpt-5-nano", logger)

CheckOpenAIWorksV2(oaiClient, cfg.InferenceBaseURL+"/openrouter", "openai/gpt-4o-mini", logger)

} else {

oaiClient := openai.NewClient(

option.WithBaseURL(cfg.OpenAIBaseURL),

option.WithAPIKey(cfg.OpenAIAPIKey),

)

CheckOpenAIWorksV2(oaiClient, cfg.OpenAIBaseURL, "gpt-4o-mini", logger)

ok. to consider informing user

@Junyi-99 oh yeah need to explicitly enable the model otherwise would get an invalid-model-id error. Do you think its wise to change it to a more commonly allowed model like 4o-mini, or should we inform the user somewhere to minimally enable gpt-5-nano.

webapp/_webapp/src/views/settings/setting-text-input.tsx

Copilot · 2026-01-24T22:33:55Z

webapp/_webapp/src/views/settings/setting-text-input.tsx

+          const matchingModel = response.models.find(m =>
+            currentSlugLower.includes(m.name.toLowerCase())


The model matching logic has a flaw: it checks if the current slug includes the model name, but this can cause incorrect matches when model names are substrings of each other. For example, if the current slug is 'openai/gpt-4.1-mini', the find will match 'GPT-4.1' first (since 'openai/gpt-4.1-mini'.includes('gpt-4.1') is true), instead of matching 'GPT-4.1-mini'. This means a user with 'gpt-4.1-mini' would incorrectly switch to 'gpt-4.1' when adding/removing BYOK. Consider matching on the exact model name by extracting the base model name from the slug (e.g., by removing the 'openai/' prefix and comparing directly).

Suggested change

const matchingModel = response.models.find(m =>

currentSlugLower.includes(m.name.toLowerCase())

const baseModelName = currentSlugLower.startsWith("openai/")

? currentSlugLower.slice("openai/".length)

: currentSlugLower;

const matchingModel = response.models.find(m =>

m.name.toLowerCase() === baseModelName

fixed. improved logic checks for suffix match.

webapp/_webapp/src/views/settings/setting-text-input.tsx

Copilot · 2026-01-24T22:33:55Z

webapp/_webapp/src/views/settings/setting-text-input.tsx

+          }
+        }
+      }
+    }, [value, settingKey, updateSettings]); // settingKey is an outer scope value, not a dependency


The dependency array is missing 'setCurrentConversation' which is used inside the callback. While it's stable (from a zustand store), it should still be included in the dependency array for completeness and to follow React best practices.

Suggested change

}, [value, settingKey, updateSettings]); // settingKey is an outer scope value, not a dependency

}, [value, settingKey, updateSettings, setCurrentConversation]); // settingKey is an outer scope value, not a dependency

not necessary. Reduce deps.

@Junyi-99 i added settingKey here as a dependency because i thought this callback should be re-created whensettingKey is updated. Was is not the case previously too?

wjiayis · 2026-01-25T07:24:43Z

This issue is probably already solved with my PR #97 (updated today 25/01), where GetDefaultParamsV2 will strip away the openai/ prefix if a custom model is specified in llmProvider.

wjiayis · 2026-01-25T08:35:34Z

Also I fixed some bugs for this PR in #100 .

4ndrelim · 2026-01-25T08:58:41Z

This issue is probably already solved with my PR #97 (updated today 25/01), where GetDefaultParamsV2 will strip away the openai/ prefix if a custom model is specified in llmProvider.

I think the primary issue is inconsistent state in currentConversation. When BYOK changes, we want the state to update the correct model (and modelSlug) as defined in listed_models.tsx which should be the source of truth. Adding this stripping layer to the backend will be adding additional coupling (backend now assumes knowledge of the state) and possibly might not be compatible for future extensions (perhaps in the future we may need to strip Qwen/ etc tho this is unlikely).

ultimately, the flow should be:

User updates BYOK at any point
Settings changes
Update conversation state to reflect modelslug
Backend uses updated modelSlug
Backend may choose to do additional safeguards, but this is secondary checks
[Consideration] maybe custom() model checks can be encoded in currentConversation state itself. But that’s up for discussion to change. For now, I don’t think it’s needed.

we can still have this additional stripping logic in the backend as a safeguard, but it should not be relied upon.

4ndrelim · 2026-01-25T08:59:23Z

Also I fixed some bugs for this PR in #100 .

Okay thanks! Can u propose merge to this branch instead? Final changes will be integrated together after testing.
Don't worry about your design changes in #97. I will rebase and resolve conflicts in this PR off your changes. This PR is primarily targeting the stale modelSlug but i may have made some intemrediary changes to the client_v2 to facilitate testing.

wjiayis · 2026-01-25T09:10:44Z

@4ndrelim Yup I proposed merge to this branch in #100

wjiayis · 2026-01-25T09:13:44Z

@4ndrelim Yea on hindsight, this is a more native solution than #97, for the same problem. Feel free to close that PR once this is merged!

1. fix: remove definition of `currentConversation` as it is not used (to fix the following error when running `npm run build:prd:chrome` ```bash > webapp@0.0.0 build:prd:chrome > bash -c 'source ./scripts/build-prd-chrome.sh' 🧩 Start building Chrome Extension ... 📦 Version: 2.16.0 📦 Monorepo Revision: 9fc435 📦 Beta Build: false 📦 API Endpoint: https://app.paperdebugger.com ❌ Failed to build Chrome Extension, please check logs/build.log > webapp@0.0.0 build > tsc -b && npm run _build:default && npm run _build:background && npm run _build:intermediate && npm run _build:settings && npm run _build:popup src/views/settings/setting-text-input.tsx(32,13): error TS6133: 'currentConversation' is declared but its value is never read. ``` 2. fix: change matching from substring to exact, to prevent `gpt-4.1-mini` from matching to `gpt-4.0` (edit: typo, should be `gpt-4.1`), making the method more robust to the order of supported models defined. 3. [edit] refactor: use IsCustom and not redefine logic in NewAIClientV2

Junyi-99

lgtm

Junyi-99 · 2026-01-26T03:26:15Z

@wjiayis Thank you for finding the bug and submitting the PR.

@4ndrelim Thank you for helping review the code and guide new contributors.💪

4ndrelim added 2 commits January 25, 2026 03:18

fix: BYOK

3ae3f49

fix: stale mdoelslug

9fc435f

Copilot AI review requested due to automatic review settings January 24, 2026 22:29

4ndrelim marked this pull request as draft January 24, 2026 22:29

Copilot started reviewing on behalf of 4ndrelim January 24, 2026 22:30 View session

4ndrelim linked an issue Jan 24, 2026 that may be closed by this pull request

[Bug] Backend retains stale modelSlug after inserting BYOK #98

Closed

Copilot AI reviewed Jan 24, 2026

View reviewed changes

wjiayis mentioned this pull request Jan 25, 2026

fix: byok compatibility #97

Closed

wjiayis and others added 4 commits January 25, 2026 22:24

nits

9f037fb

nits

81f2457

improve matching modelSlug logic

7deb2c4

4ndrelim requested review from Junyi-99 and kah-seng January 25, 2026 16:32

4ndrelim marked this pull request as ready for review January 25, 2026 16:32

Junyi-99 approved these changes Jan 26, 2026

View reviewed changes

Junyi-99 merged commit 5b36033 into main Jan 26, 2026
2 checks passed

Junyi-99 deleted the fix-stale-modelslug branch January 26, 2026 03:26

		const matchingModel = response.models.find(m =>
		currentSlugLower.includes(m.name.toLowerCase())

-          const matchingModel = response.models.find(m =>
-            currentSlugLower.includes(m.name.toLowerCase())
+          const baseModelName = currentSlugLower.startsWith("openai/")
+            ? currentSlugLower.slice("openai/".length)
+            : currentSlugLower;
+          const matchingModel = response.models.find(m =>
+            m.name.toLowerCase() === baseModelName

	}, [value, settingKey, updateSettings]); // settingKey is an outer scope value, not a dependency
	}, [value, settingKey, updateSettings, setCurrentConversation]); // settingKey is an outer scope value, not a dependency

fix: stale modelSlug #99

fix: stale modelSlug #99

Conversation

4ndrelim commented Jan 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Jan 24, 2026

Choose a reason for hiding this comment

Uh oh!

4ndrelim Jan 24, 2026

Choose a reason for hiding this comment

Uh oh!

4ndrelim Jan 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Jan 24, 2026

Choose a reason for hiding this comment

Uh oh!

4ndrelim Jan 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Jan 24, 2026

Choose a reason for hiding this comment

Uh oh!

4ndrelim Jan 24, 2026

Choose a reason for hiding this comment

Uh oh!

4ndrelim Jan 25, 2026

Choose a reason for hiding this comment

Uh oh!

wjiayis commented Jan 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wjiayis commented Jan 25, 2026

Uh oh!

4ndrelim commented Jan 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

4ndrelim commented Jan 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wjiayis commented Jan 25, 2026

Uh oh!

wjiayis commented Jan 25, 2026

Uh oh!

Junyi-99 left a comment

Choose a reason for hiding this comment

Uh oh!

Junyi-99 commented Jan 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

4ndrelim commented Jan 24, 2026 •

edited

Loading

4ndrelim Jan 25, 2026 •

edited

Loading

wjiayis commented Jan 25, 2026 •

edited

Loading

4ndrelim commented Jan 25, 2026 •

edited

Loading

4ndrelim commented Jan 25, 2026 •

edited

Loading