Skip to content

Pull requests: OpenRouterTeam/benchmark-harness

Pull request creation is restricted
12 Open 71 Closed
12 Open 71 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

fix(harness): treat HTTP 408 timeouts as retryable and score exhausted timeouts as Skipped
#83 opened Sep 11, 2026 by devin-ai-integration Bot Loading…
3 of 5 tasks
1
deps: bump the dependencies group across 1 directory with 18 updates
#82 opened Sep 10, 2026 by dependabot Bot Loading…
1
fix(cli): forward solver config to all model benchmarks and improve error resilience
#77 opened Sep 8, 2026 by devin-ai-integration Bot Loading…
3 of 5 tasks
1
feat(agent-cli): add ori code harness for terminal-bench
#76 opened Sep 8, 2026 by devin-ai-integration Bot Loading…
3 of 5 tasks
@louisgv
1
feat(agent-cli): add codex, opencode and kilo ori harnesses with in-sandbox generation-id proxy
#73 opened Sep 3, 2026 by abhinav-pola Contributor Loading…
3 of 5 tasks
@abhinav-pola
11
feat: add Terminal-Bench 4.0 as a separate Modal-backed benchmark
#72 opened Sep 3, 2026 by abhinav-pola Contributor Loading…
5 tasks done
@abhinav-pola
21
feat: persist completed sample outcomes so interrupted runs resume instead of restarting
#48 opened Aug 21, 2026 by jamespsterling Contributor • Draft
4 of 5 tasks
@jamespsterling
33
fix(search): resolve server-tool generation ids
#32 opened Aug 14, 2026 by ayush-or Contributor Loading…
feat(benchmarks): chess — full games vs Stockfish, first-party original
#4 opened Aug 5, 2026 by LukasParke Loading…
2
feat(benchmarks): EvalManifestV1 — unified eval description schema + first-party manifests
#3 opened Aug 5, 2026 by LukasParke Loading…
2
feat(benchmarks): custom_eval — declarative user evals as run-config data
#2 opened Aug 5, 2026 by LukasParke Loading…
2
feat(harness): durable per-(sample, epoch) result store + fiber-scoped request context
#1 opened Aug 5, 2026 by LukasParke Loading…
3
ProTip! Filter pull requests by the default branch with base:main.