feat(litellm): add flatten content option for auto_router compatibility #11133
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR attempts to address Issue #11132.
Problem
LiteLLM's
auto_routerfeature passes user content to embedding models for semantic routing. However, embedding models expect plain strings, not arrays of content blocks. When Roo Code sends messages with array content like:{"role": "user", "content": [{"type": "text", "text": "..."}]}The auto_router fails because it passes this array directly to embeddings, which is incompatible.
Solution
This PR adds a new
litellmFlattenContentoption (enabled by default) that flattens array content to simple string format before sending to LiteLLM:{"role": "user", "content": "..."}Changes
litellmFlattenContentoption to the LiteLLM schemaflattenMessageContent()method and apply it increateMessage()when the option is enabled (default: true)Notes
Feedback and guidance are welcome!
Important
Adds
litellmFlattenContentoption to flatten message content for LiteLLM'sauto_router, with schema, UI, and test updates.litellmFlattenContentoption to flatten array content to strings for LiteLLM'sauto_routercompatibility.litellmFlattenContentis set to false.litellmFlattenContenttoprovider-settings.tsschema.flattenMessageContent()inlite-llm.tsand applies it increateMessage().LiteLLM.tsx.lite-llm.spec.tsto verify flattening logic under different configurations.settings.jsonin 17 locales to include translations for the new option.This description was created by
for 6fe9a9e. You can customize this summary. It will automatically update as commits are pushed.