Skip to content

ci: integrate callgrind benchmarks using iai-callgrind - #675

Merged
alejandro-vaz merged 5 commits into
servo:v2from
Zul-Qarnain:feat/callgrind-ci
Sep 25, 2026
Merged

alejandro-vaz merged 5 commits into
servo:v2from
Zul-Qarnain:feat/callgrind-ci

Conversation

@Zul-Qarnain

Copy link
Copy Markdown
Contributor

This PR adds instruction-level benchmarks using iai-callgrind and integrates them into CI to catch regressions deterministically:

  • Adds benches/callgrind.rs testing core vector operations (push, pop, insert/remove, slice conversions, drain, retain_mut, into_iter) across both inline and spilled heap storage.
  • Adds a callgrind job to .github/workflows/checks.yml running the benchmarks on both default features and --all-features.

Comment thread .github/workflows/checks.yml Outdated
@alejandro-vaz alejandro-vaz linked an issue Sep 25, 2026 that may be closed by this pull request
@alejandro-vaz

Copy link
Copy Markdown
Collaborator

CI shows nothing

callgrind::push_group::bench_smallvec_push_inline
  Instructions:                           0|N/A                  (*********)
  L1 Hits:                                0|N/A                  (*********)
  L2 Hits:                                0|N/A                  (*********)
  RAM Hits:                               0|N/A                  (*********)
  Total read+write:                       0|N/A                  (*********)
  Estimated Cycles:                       0|N/A                  (*********)

@alejandro-vaz alejandro-vaz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it doesn't seem to be working properly

@Zul-Qarnain

Copy link
Copy Markdown
Contributor Author

Fixed in 5cd5e5b.

The 0|N/A output was because cargo bench built the callgrind target with [profile.bench]'s debug = false + strip = true, so Callgrind had no function symbols to attribute events to and collected 0 for every benchmark.

I added a dedicated [profile.callgrind] (inherits bench, but debug = 1, strip = false, lto = false) and pointed both CI steps at it via --profile callgrind.

Verified on the same runner (fork Actions run): instruction counts are now non-zero, e.g. push_inline=361, push_spilled=2952, vec_push=120, pop_inline=484.

Note: the PR's own workflow is still sitting at the "Approve and run workflows" gate for fork PRs — could someone approve it so the callgrind check runs here too?

@alejandro-vaz

Copy link
Copy Markdown
Collaborator

does LTO have to be disabled in order for it to work??

@Zul-Qarnain

Copy link
Copy Markdown
Contributor Author

No. I tested lto = "fat", "thin", and false (plus codegen-units 1 vs 16) across all 19 benchmarks — every config gives correct, non-zero counts within ~1% of each other (e.g. push_inline = 361 in all cases).

The actual requirement is just strip = false so Callgrind keeps the symbol table; neither debug nor lto affects whether events are collected. I've dropped lto = false in 701a661 so the callgrind run stays representative of the normal bench codegen.

@alejandro-vaz

Copy link
Copy Markdown
Collaborator

thanks for contributing

@alejandro-vaz
alejandro-vaz added this pull request to the merge queue Sep 25, 2026
Merged via the queue into servo:v2 with commit 1d55ab4 Sep 25, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

integrate callgrind to CI

2 participants