Importing `vortex` after `pyarrow.dataset` impacts the `pyarrow` runtime, causing SIGSEGV

[alexander-beedie]

What happened?

Importing vortex after importing pyarrow.dataset corrupts pyarrow's runtime state.
Calling in to pyarrow can then segfault.

Steps to reproduce

import pyarrow.dataset
import vortex
import pyarrow as pa

pa.array([1])

# Process finished with exit code 139 (interrupted by signal 11:SIGSEGV)

Environment

• Vortex version 0.70.0
• Python: 3.11
• macOS: 26.4.1

Suspected root cause

Maybe vortex's Python extension statically links its own copy of the Arrow runtime? If imported after pyarrow.dataset, this means parts of Arrow's process-global state could be overwritten, with subsequent pyarrow calls dereferencing function pointers or type handles that point into a runtime it's not coordinated with.

Possible fix

Reuse the existing pyarrow runtime when it's already loaded?
Ref: pyarrow C++ integration docs.

Additional context

Reversing the import order lets pyarrow's registrations win, leaving the runtime self-consistent:

• pyarrow.dataset → vortex

  

• vortex → pyarrow.dataset

  

[joseph-isaacs]

#7826

[alexander-beedie]

Ahhh, nice - not exactly what I suspected, but great there's a clean fix.
Thanks 😎👍