Skip to content

coreml_static_int8 operators suite never finishes on macOS (hits the 180 min cap) #21623

Description

@shoumikhin

The coreml_static_int8 half of test-backend-macos (operators suite) has never completed. It runs to the 180 minute job cap and gets cancelled, so it produces no signal at all.

Measured on a recent main commit:

flow / suite duration
coreml / models 12 min
coreml / operators 11 min
coreml_static_int8 / models 25 min
coreml_static_int8 / operators 181 min, cancelled at the cap

Both flows are built by the same _create_coreml_flow in backends/test/suite/flows/coreml.py. coreml_static_int8 is the identical models + operators suite with quantize=True and minimum_deployment_target=iOS17. So the cost is a full PT2E prepare / calibrate / convert pass on every single test case, not larger weights.

#21609 moved this flow to the nightly schedule, so it no longer consumes the shared macOS pool on every commit. That limits the damage but does not fix it: the nightly run will keep hitting the same cap, and the int8 flow will keep reporting nothing.

Possible directions, roughly in order of effort:

  • Shard the operators suite across several jobs so each one finishes inside the cap.
  • Reuse a single quantized artifact across cases where the operator under test allows it, instead of quantizing per case.
  • Trim the int8 case list to the operators where quantization actually changes the lowering.

Until one of those lands, the int8 operators coverage should be treated as absent rather than passing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions