[Bug]: `model_supports_json` is no longer supported on v3, breaking custom model setups

[Voileexperiments]

Do you need to file an issue?

• I have searched the existing issues and this bug is not already filed.
• My model is hosted on OpenAI or Azure. If not, please look at the "model providers" issue and don't file a new one here.
• I believe this is a legitimate bug, not just a question. If this is a question, please use the Discussions area.

Describe the bug

Most models do not support structured output. In this case the LLM caller (liteLLM) usually falls back to a tool call. However, a tool call will cause response content to be empty, which then the LLM caller tries to parse, and fail.

In v2 there is a model_supports_json param, which can be disabled to not use structured output in the LLM calls, to avoid this issue. In v3 this setting is gone, and so every model that doesn't support structured output will fail to work.

Steps to reproduce

No response

Expected Behavior

No response

GraphRAG Config Used

# Paste your config here

Logs and screenshots

No response

Additional Information

• GraphRAG Version: 3.0.5

[JiwaniZakir]

The removal of model_supports_json in v3 means the LLM caller always attempts structured output, and when the model falls back to a tool call the response content comes back empty, causing the parse failure. I can restore this as a config option in the chat LLM integration so it skips structured output when disabled, similar to how v2 handled it. I'll dig into the LLM caller to make sure the tool call response path also gets handled gracefully when structured output isn't available.

[JiwaniZakir]

Stepping back from this one — the fix was more involved than expected and I ran out of time. Free for anyone else.

[dworthen]

Hi @Voileexperiments. We added client side json validation support according to LiteLLM documentation (https://docs.litellm.ai/docs/completion/json_mode#validate-json-schema) in PR #2306 but have not been able to fully test as all of our test models support structured responses. Can you provide more details on how to replicate? Which models are you using? Can you provide the configuration file?