Improve local Ollama fallback for low-resource machines#1549
Open
naraypv wants to merge 4 commits into
Open
Conversation
# Conflicts: # README.md # graphify/build.py # graphify/detect.py # graphify/llm.py # graphify/prs.py # graphify/skill-amp.md # graphify/skill-claw.md # graphify/skill-codex.md # graphify/skill-copilot.md # graphify/skill-droid.md # graphify/skill-kilo.md # graphify/skill-kiro.md # graphify/skill-opencode.md # graphify/skill-pi.md # graphify/skill-trae.md # graphify/skill-vscode.md # graphify/skill-windows.md # graphify/skill.md # tests/test_detect.py # tools/skillgen/expected/graphify__skill-amp.md # tools/skillgen/expected/graphify__skill-claw.md # tools/skillgen/expected/graphify__skill-codex.md # tools/skillgen/expected/graphify__skill-copilot.md # tools/skillgen/expected/graphify__skill-droid.md # tools/skillgen/expected/graphify__skill-kilo.md # tools/skillgen/expected/graphify__skill-kiro.md # tools/skillgen/expected/graphify__skill-opencode.md # tools/skillgen/expected/graphify__skill-pi.md # tools/skillgen/expected/graphify__skill-trae.md # tools/skillgen/expected/graphify__skill-vscode.md # tools/skillgen/expected/graphify__skill-windows.md # tools/skillgen/expected/graphify__skill.md # tools/skillgen/fragments/core/core.md # uv.lock
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR improves Graphify's local semantic extraction path for machines that cannot comfortably run large local models.
Implemented changes:
qwen2.5-coder:3bfirst, thengemma3:4b/api/chatendpoint so context options such asnum_ctxactually applyWhy this may help the community
A lot of users do not have enough RAM/VRAM to run 20B or 30B coding models while also using their machine for normal work. This keeps the local path useful on lower-resource laptops by trying small local models first, failing over cleanly, and only using cloud fallback when the user has configured it. It should make large-project Graphify runs less brittle without forcing everyone into large local model downloads.
Verification
Ran locally before opening this PR:
uv run pytest -q-> 2537 passed, 28 skippedpython3 -m compileall -q graphifypython3 -m tools.skillgen --checkuv lock --checkqwen2.5-coder:3band producedgraph.jsongraphify update . --no-clusterHEADNo repo-specific CONTRIBUTING file or pull request template was present, so this targets the upstream default branch
v8and follows the visible project/test/security expectations.