NXP backend: handle tests of mlperf tiny visual wake words

[novak-vaclav]

Summary

Enable and refactor MLPerf Tiny Visual Wake Words tests.

Test plan

tests can be manually run using pytest -c /dev/null backends/nxp/tests/

cc @robert-kalmar @JakeStevens @digantdesai @rascani @roman-janik-nxp

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22844

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 23 New Failures, 1 Pending, 2 Unclassified Failures

As of commit 4b4dac8 with merge base b0bc7b2 ():

NEW FAILURES - The following jobs have failed:

• periodic / test-models-linux (buck2, mv3, portable, mt-l-x86iavx512-8-64, 90) / linux-job (gh)
  Could not load credentials from any providers
• periodic / test-models-linux (cmake, vit, portable, mt-l-x86iavx512-8-64, 90) / linux-job (gh)
  Could not load credentials from any providers
• pull / test-arm-backend-no-driver (test_run_tosa) / linux-job (gh)
  Could not load credentials from any providers
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  Could not load credentials from any providers
• pull / test-mcu-cortex-m-backend / linux-job (gh)
  Could not load credentials from any providers
• pull / test-models-linux (emformer_join, portable, mt-l-x86iavx512-16-128) / linux-job (gh)
  Could not load credentials from any providers
• pull / test-models-linux (ic3, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job (gh)
  Could not load credentials from any providers
• pull / test-models-linux (ic4, xnnpack-quantization-delegation, mt-l-x86iavx512-16-128) / linux-job (gh)
  Could not load credentials from any providers
• pull / test-models-linux (linear, portable, mt-l-x86iavx512-8-64) / linux-job (gh)
  Could not load credentials from any providers
• pull / test-models-linux (w2l, portable, mt-l-x86iavx512-16-128) / linux-job (gh)
  Could not load credentials from any providers
• pull / test-models-linux-basic (mv3, portable, cmake, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clan... / linux-job (gh)
  Could not load credentials from any providers
• pull / test-models-linux-basic (vit, portable, buck2, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clan... / linux-job (gh)
  Could not load credentials from any providers
• pull / test-models-linux-basic (vit, portable, cmake, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clan... / linux-job (gh)
  Could not load credentials from any providers
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, mt-l-x86iavx512-8-64, execu... / linux-job (gh)
  Could not load credentials from any providers
• pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, mt-l-x86iavx512-8-64, execu... / linux-job (gh)
  Could not load credentials from any providers
• pull / test-moshi-linux / linux-job (gh)
  Could not load credentials from any providers
• pull / test-qnn-direct-build-linux / linux-job (gh)
  Could not load credentials from any providers
• pull / test-qnn-testsuite-linux / test-backend-linux (qnn, models) / linux-job (gh)
  Could not load credentials from any providers
• pull / test-qnn-wheel-packages-linux (3.10) / linux-job (gh)
  Could not load credentials from any providers
• pull / test-qnn-wheel-packages-linux (3.13) / linux-job (gh)
  Could not load credentials from any providers
• pull / test-quantized-aot-lib-linux / linux-job (gh)
  Could not load credentials from any providers
• pull / test-voxtral-realtime-xnnpack-linux / linux-job (gh)
  Could not load credentials from any providers
• Test WebGPU Backend / test-webgpu / test-backend-linux (webgpu, operators) / linux-job (gh)
  Could not load credentials from any providers

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

• Build Aarch64 Linux CUDA Wheels / pytorch/executorch / build-wheel-py3_10-cuda-aarch6413_0-aarch64 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ##[error]fatal: clone of 'https://gitlab.com/libeigen/eigen.git' into submodule path '/__w/executorch/executorch/pytorch/executorch/kernels/optimized/third-party/eigen' failed
• Build Aarch64 Linux CUDA Wheels / pytorch/executorch / upload / upload-wheel-py3_10-cuda-aarch6413_0-aarch64 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  Unable to download artifact(s): Artifact not found

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[linux-foundation-easycla]

• ❌ - login: @novak-vaclav / name: Vaclav Novak. The commit (4b4dac8) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please visit our EasyCLA portal and chat with our support bot.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[novak-vaclav]

Fixed invalid bound.

[roman-janik-nxp]

No adding the model to aot_neutron_compile.py and tests for it?

[roman-janik-nxp]

Nit: I would keep the same order of parameters in all tests (and previous one).

[novak-vaclav]

I checked params in lower_run_compare and lower_run_compare_ptq_qat in all executorch MLPerf Tiny tests to make them correspond to the order in your Anomaly Detection PR. The only issue was in image classification.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[novak-vaclav]

You are right @roman-janik-nxp, I totally forgot about the aot_neutron_compile and its test. I added it now. Thank you.
Now it is ready for review.