zephyr-cp: run the tests in parallel - #11348
Conversation
4da0a0f to
3410804
Compare
tannewt
left a comment
tannewt
left a comment
There was a problem hiding this comment.
Thanks! One question about CPU count on actions VMs.
| test: $(TEST_BOARDS:%=test-build-%) | ||
| pytest cptools/tests | ||
| pytest tests/ -v | ||
| pytest tests/ -v -n 2 --dist loadfile |
There was a problem hiding this comment.
Aren't most VMs now 4 CPU? Is there a way for use to pass the correct number in?
There was a problem hiding this comment.
I think this should just be -v $(nproc) like the others
There was a problem hiding this comment.
Originally, I had -n $$(nproc || sysctl -n hw.ncpu).
The run was https://github.com/adafruit/circuitpython/actions/runs/34614129245 - the suite finished in 7:38, but five bsim tests failed with Low level communication with phy failed. A bsim test runs a phy and one or two simulated devices, so with four workers a dozen CPU bound processes can share the four vCPUs (it's probably just 2 physical cores with HT). Two workers pass locally; the run on 2 is going now.
There was a problem hiding this comment.
Maybe split the bsim tests from the non-bsim?
There was a problem hiding this comment.
The split has already been implemented in PR #11350. I still need to investigate this further to see if it's a dead end... It worked fine on my local machine 🙂
There was a problem hiding this comment.
I think I've found the root cause of those build issues. This PR might be back in the game after PR #11353.
Not for merge as it stands: pytest-xdist is installed in the job rather than in requirements-dev.txt, so that this change touches only a file in IGNORE_BOARD and the run builds no boards. The question is whether the bsim tests stay green when the only thing they compete with is another bsim simulation. Earlier runs parallelised the whole suite, where the asan native tests ran alongside: four workers lost five tests, two workers lost one.
3410804 to
ce4ca59
Compare
3 participants
make testruns the suite one test at a time. It is 17 of the zephyr-tests job's 34 to 44 minutes, and 84% of that is bsim, where every test runs a phy and one or two simulated devices.pytest now runs with one worker per CPU.
--dist loadfilekeeps each file on one worker, whichtest_web_workflow.pyneeds for its fixed port 8090.Measured locally on four cores, whole suite, 322 passed each time: 758 s with one worker, 350 s with two, 249 s with three, 224 s with four. -n auto asks psutil for physical cores, which inside a container is the host's count. If the bsim tests need more room on the runner, the count is one edit.