docs(examples): add benchSDK example benchmarks - #308
Open
dtice25 wants to merge 3 commits into
Open
Conversation
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Contributor License AgreementAll contributors are covered by a CLA. |
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ingest Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
devin-ai-integration
Bot
force-pushed
the
devin/examples
branch
from
August 18, 2026 16:37
91d96d6 to
77d3e76
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rebases the examples branch onto the latest
master(which added per-stepconcurrency/timeoutMs, theonScorehook, and--no-ingestdry-run mode) and updates the examples to cover those new primitives.What's changed
devin/examplesontoorigin/masterto pick up recent@benchsdk/runnerchanges.benchmarks/examples/05-step-options.bench.ts— demonstratesctx.step(..., { concurrency: N, timeoutMs: N })for parallel step invocations.benchmarks/examples/06-scoring.bench.ts— demonstratesconfig.onScore,lowerIsBetter, and the exportedscore()helper for weighted composite scoring.benchmarks/examples/README.mdwith--no-ingest/BENCHSDK_NO_INGESTinstructions, the new examples, and corrected output expectations.--no-ingestnotes to the header comments of the existing examples so they can be run locally without platform credentials.Verification
pnpm -r --filter "./packages/**" buildsucceeded.pnpm typecheckpassed.--no-ingestmode, including the new step-options and scoring examples.Link to Devin session: https://app.devin.ai/sessions/770c96fe577f4323ae5d7f8ca8b785dc
Requested by: @dtice25