feat(ci): implement sharding for unit test performance - #9021
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request parallelizes test execution and adds sharding support in the CI conditional test runner, configures the Mocha reporter and silences pnpm installation in the single test runner, and updates mock discovery URLs in the googleapis-common tests. A critical syntax error was identified in the conditional test runner script, where leftover loop and conditional control structures will cause the Bash script to fail.
bshaffer
changed the title
|
@gemini-code-assist review |
There was a problem hiding this comment.
Code Review
This pull request introduces test sharding and dynamic matrix generation for GitHub Actions in the CI scripts, updates pnpm install flags, and transitions from the deprecated url.parse to the standard URL API. Additionally, it increases test timeouts, skips certain tests on Windows, handles nested error causes in gcp-metadata, and ensures startTime does not exceed endTime in Bigtable metrics. Feedback focuses on improving shell script robustness by quoting variables (BUILD_TYPE, d, and test_script) to prevent syntax or word-splitting issues, and avoiding direct mutation of the options parameter in the OpenTelemetry test stub.
westarle
left a comment
westarle
left a comment
There was a problem hiding this comment.
A few comments, I think this is a really great idea!
westarle
left a comment
westarle
left a comment
There was a problem hiding this comment.
Also, would you be OK with breaking the test fixes out to their own little PRs?
bshaffer
changed the title
0e8030b to
8015ceb
Compare
…-n-play test timeouts
67082d7 to
657c33a
Compare
657c33a to
f18b9c1
Compare
bshaffer
changed the title
chore(ci): reduce unit-test output due to log truncation chore: run all ci tests
f18b9c1 to
66743ae
Compare
2 participants
Conditional Test Sharding: Dynamically shards tests when a large number of packages are modified. Adds a job that rolls up shard results to satisfy branch protection. Uses "composite" actions to deduplicate workflow YAML between unit and windows unit tests. For what this looks like with just ONE package modified, see chore: test sharding with one package #9027
Run the full unit test suite: Restores triggering the full unit test suite across all packages whenever
ci/scripts change, as sharding now allows running all tests without timing out.Clean CI Logs: Switches Mocha to use
dotreporter and removes accidentalconsole.logfrom the Firestore conformance tests.Depends on #9061