Use AllNonDistinct in assert_arrays_eq and implement it for variant types

[robert3005]

AllNonDistinct is exactly the semantics we want, it should be faster and in case
of failure we run the slow method to produce precise error message

[codspeed-hq]

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 1 improved benchmark
❌ 2 regressed benchmarks
✅ 1582 untouched benchmarks
⏩ 4 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	chunked_varbinview_canonical_into[(100, 100)]	224.4 µs	259.5 µs	-13.52%
❌	Simulation	chunked_varbinview_into_canonical[(100, 100)]	271.4 µs	306.6 µs	-11.48%
⚡	Simulation	chunked_varbinview_into_canonical[(1000, 10)]	205.6 µs	168.9 µs	+21.76%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing assert_arrays_eq_allnondistinct (828662a) with develop (f41dc23)}

Footnotes

1. 4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩