Skip to content

replace openmatrix with h5py - #87

Open
jpn-- wants to merge 4 commits into
ActivitySim:mainfrom
driftlesslabs:codex/replace-openmatrix-with-h5py
Open

replace openmatrix with h5py#87
jpn-- wants to merge 4 commits into
ActivitySim:mainfrom
driftlesslabs:codex/replace-openmatrix-with-h5py

Conversation

@jpn--

@jpn-- jpn-- commented Aug 7, 2026

Copy link
Copy Markdown
Member

This pull request removes the use of the openmatrix package throughout the codebase and replaces it with direct usage of the h5py library for reading and writing OMX (Open Matrix) HDF5 files. The change simplifies dependencies, updates documentation and examples, and refactors code to use h5py APIs. Additionally, improvements are made to shared memory handling, including support for custom array storage order and better memory management.

Dependency and Documentation Updates:

  • Removed openmatrix from all dependency files (envs/development.yml, envs/testing.yml, pyproject.toml, .github/workflows/run-tests.yml) and updated code, documentation, and examples to use h5py directly for OMX file operations. [1] [2] [3] [4] [5] [6] [7] [8] [9]

Code Refactoring:

  • Refactored all code that previously used openmatrix to use h5py, including in sharrow/example_data.py, sharrow/omx.py, and sharrow/tests/test_datasets.py. This includes updating file reading/writing logic and dataset creation. [1] [2] [3] [4] [5]

OMX File Handling Improvements:

  • Rewrote the split_omx function in sharrow/omx.py to use h5py for splitting OMX files, added helper functions for copying metadata and datasets, and improved output file organization and lookup management.

Shared Memory Enhancements:

  • Added support for specifying custom array storage order (e.g., "last-axis-first") in shared memory, improved buffer initialization to be more memory efficient, and enhanced cleanup logic for memory-mapped arrays in sharrow/shared_memory.py. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

API and Type Annotations:

  • Updated function signatures and docstrings to reflect the removal of openmatrix and clarify the use of h5py objects, especially in sharrow/omx_reader.py. [1] [2]

jpn-- added 4 commits August 5, 2026 23:33
Reopen compatible OMX wrappers through h5py while preserving caller ownership.

Add coverage for 2D, 3D, and reload workflows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant