Skip to content

🏃 add input + combined startup benchmarks#56

Merged
natemoo-re merged 2 commits into
mainfrom
bench/combined
Jun 3, 2026
Merged

🏃 add input + combined startup benchmarks#56
natemoo-re merged 2 commits into
mainfrom
bench/combined

Conversation

@natemoo-re
Copy link
Copy Markdown
Member

@natemoo-re natemoo-re commented Jun 1, 2026

Follow-up to #52. Adds new startup benchmarks in mode: walltime to measure the full E2E perf of:

  • createInput bootstrap
  • combined createTerm + createInput bootstrap inside Promise.all
  • render + scan hitting both modules

also sets both benchmark jobs to codspeed-macro runner to avoid hardware variance, which was causing quite a bit of noise

@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new Bot commented Jun 1, 2026

Open in StackBlitz

npm i https://pkg.pr.new/clayterm@56

commit: 2fbe488

@codspeed-hq
Copy link
Copy Markdown
Contributor

codspeed-hq Bot commented Jun 1, 2026

Merging this PR will degrade performance by 37.29%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 2 (👁 2) regressed benchmarks
✅ 18 untouched benchmarks
🆕 3 new benchmarks

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 WallTime createInput N/A 283 ms N/A
🆕 WallTime render + scan (both modules) N/A 374 ms N/A
🆕 WallTime createTerm + createInput N/A 374.5 ms N/A
👁 Simulation long input burst (200 bytes) 979.3 µs 1,265.3 µs -22.61%
👁 Simulation printable ASCII (single char) 88.7 µs 174.5 µs -49.18%

Comparing bench/combined (2fbe488) with main (ebaf397)

Open in CodSpeed

@cowboyd
Copy link
Copy Markdown
Collaborator

cowboyd commented Jun 3, 2026

❌ 2 (👁 2) regressed benchmarks

Seeing as this did not actually touch any library code, feels like either we aren't running our benchmarks enough, or we need to run them not in absolute terms but in relative to a control that can account for fluctuations in runtime.

@natemoo-re
Copy link
Copy Markdown
Member Author

@cowboyd yeah this PR also switched the runners to codspeed-macro which will be consistent—ubuntu-latest varies wildly and was creating a lot of noise. thankfully this should be the last noisy one!

@natemoo-re natemoo-re merged commit d75fb4b into main Jun 3, 2026
11 checks passed
@natemoo-re natemoo-re deleted the bench/combined branch June 3, 2026 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants