Skip to content

Add CUDA decompression kernels for OnPair - #8920

Open
robert3005 wants to merge 14 commits into
developfrom
rk/onpairgpu
Open

Add CUDA decompression kernels for OnPair#8920
robert3005 wants to merge 14 commits into
developfrom
rk/onpairgpu

Conversation

@robert3005

Copy link
Copy Markdown
Contributor

They follow the same pattern as FSST where we let users choose varbin/varbinview

@robert3005
robert3005 requested review from 0ax1 and joseph-isaacs July 23, 2026 14:33
Comment thread vortex-cuda/benches/onpair_cuda.rs Outdated
Comment thread vortex-cuda/src/kernel/encodings/onpair.rs Outdated
@codspeed-hq

codspeed-hq Bot commented Jul 23, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 30.31%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 3 improved benchmarks
✅ 1980 untouched benchmarks
⏩ 46 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation decode_varbin[(1000, 4)] 99.3 µs 60.1 µs +65.1%
Simulation cold_misaligned[(64, 256)] 5.3 ms 4.4 ms +20.73%
Simulation compress_fsst[(1000, 64, 8)] 1.2 ms 1 ms +11%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing rk/onpairgpu (47ad0ab) with develop (b4b8f54)

Open in CodSpeed

Footnotes

  1. 46 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Comment thread vortex-cuda/cub/src/scan.rs Outdated
Comment thread vortex-cuda/kernels/src/onpair_shmem_4tpt_split8read.cu
Comment thread vortex-cuda/kernels/src/onpair_shmem_4tpt_split8read.cu Outdated
Comment thread vortex-cuda/cub/kernels/filter.cu Outdated
@robert3005

Copy link
Copy Markdown
Contributor Author

I made it use the same machinery like #8921 to avoid triple kernel dispatch

@0ax1 0ax1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. but worth double checking all index types

Comment thread vortex-cuda/benches/bench_config/mod.rs
Comment thread vortex-cuda/kernels/src/onpair_shmem_4tpt_split8read.cu Outdated
Comment thread vortex-cuda/kernels/src/onpair_shmem_4tpt_split8read.cu Outdated
Comment thread vortex-cuda/kernels/src/onpair.cu Outdated
Comment thread vortex-cuda/kernels/src/onpair.cu Outdated
Comment thread vortex-cuda/src/kernel/encodings/onpair.rs
Comment thread vortex-cuda/src/kernel/encodings/onpair.rs Outdated

@joseph-isaacs joseph-isaacs left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nno no

Comment thread vortex-cuda/src/kernel/encodings/onpair.rs Outdated
Comment thread vortex-cuda/src/kernel/encodings/onpair.rs Outdated
@robert3005

Copy link
Copy Markdown
Contributor Author

I am reworking this to make sure all the work happens in one place - there's too many conflation. Will make sure that all non trivial compute is on gpu

Comment thread vortex-cuda/src/kernel/encodings/onpair.rs Outdated
Comment thread vortex-cuda/src/kernel/encodings/onpair.rs Outdated
Comment thread vortex-cuda/src/kernel/encodings/onpair.rs Outdated
@robert3005

Copy link
Copy Markdown
Contributor Author

@joseph-isaacs have another look

@robert3005
robert3005 requested a review from joseph-isaacs July 31, 2026 15:39
Comment thread vortex-cuda/src/kernel/encodings/onpair.rs
Comment thread vortex-cuda/src/kernel/encodings/onpair.rs Outdated
Comment thread vortex-cuda/src/kernel/encodings/onpair.rs Outdated
@robert3005

Copy link
Copy Markdown
Contributor Author

I have added a feature to let you spawn work on another stream here #9234. We can then rebase this

robert3005 and others added 2 commits August 10, 2026 17:16
fork() creates a sibling execution context on a stream from the session's
round-robin pool with a fresh CPU ExecutionCtx, so independent decode work
can be submitted concurrently from one executor. wait_for() orders one
context's stream behind another with a CUDA event, without blocking the
host — needed when a forked context's output is consumed through raw
device pointers (e.g. the CUB shims), which bypass cudarc's cross-stream
buffer guards.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MjcUksDnnBeoKUC8t3Twea
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Drops the sibling-stream fork/wait_for API and decodes the OnPair
children, dictionary staging, and row-offset scans sequentially on the
caller's stream.

Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants