RowFn binary numeric ScalarFn rewrite - #9320
Conversation
Merging this PR will degrade performance by 13.72%
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
51098ea to
37eb039
Compare
|
Local wall-time results do not reproduce the broad CodSpeed simulation regressions. Ratios below are For example, locally (on my machine) the comparison commit improves the Measurement types
Primitive comparison resultsBase is the numeric-only commit
Ten cases are at or below Comparison controlsBase is
The process medians ranged from 180.0 to 195.3 µs for the base Primitive numeric resultsBase is the RowFn API commit
The numeric port is mixed. Two cases are below Numeric controlsBase is
The base and candidate ranges for Cross-benchmark sensitivityThe eight-run full matrix gave This is also why the tables separate isolated measurements from controls and unstable controls. Method
These are local wall-time measurements. They are not CodSpeed CPU-simulation measurements and do not explain CodSpeed cache or instruction-cost changes. ObservationsThere are only a few numeric arithmetic regressions of at most 10%, and they all are related to 64-bit operations being harder for the compiler to optimize. If I can't find a way to undo this regression via the But in general, everything else is at parity or obviously better. |
37eb039 to
32ad0bf
Compare
RowFn Binary Numeric Implementations
RowFn Binary Numeric ImplementationsRowFn binary numeric ScalarFn rewrite
|
To help with the regressions, we could have support a mode that gives you constant size blocks of data instead of just row. But that can come later! |
|
@joseph-isaacs I already have batch execution built in to the machinery driving the RowFn framework, and also if we did that it would make it harder to define constant handling and null propagation semantics. I am pretty certain that the regressions come from 1) initialization costs that might not need to be paid (this is why we have the |
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Benchmarks: TPC-H SF=1 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (1.002x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.012x ➖, 1↑ 2↓)
duckdb / vortex-compact / ns (0.993x ➖, 1↑ 0↓)
duckdb / parquet / ns (1.003x ➖, 0↑ 0↓)
File Size Changes (9 files changed, -56.1% overall, 0↑ 9↓)
Totals:
|
Benchmarks: TPC-DS SF=1 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (1.003x ➖, 2↑ 2↓)
datafusion / parquet / ns (1.000x ➖, 0↑ 1↓)
duckdb / vortex-compact / ns (1.008x ➖, 3↑ 9↓)
duckdb / parquet / ns (0.992x ➖, 5↑ 3↓)
File Size Changes (25 files changed, -56.5% overall, 0↑ 25↓)
Totals:
|
Benchmarks: FineWeb NVMe 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (0.993x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.981x ➖, 1↑ 0↓)
duckdb / vortex-compact / ns (0.999x ➖, 1↑ 0↓)
duckdb / parquet / ns (1.013x ➖, 0↑ 0↓)
File Size Changes (2 files changed, -53.7% overall, 0↑ 2↓)
Totals:
|
Benchmarks: Clickbench Sorted on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (1.009x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.963x ➖, 1↑ 0↓)
duckdb / vortex-compact / ns (1.033x ➖, 0↑ 2↓)
duckdb / parquet / ns (0.992x ➖, 1↑ 0↓)
File Size Changes (201 files changed, -57.2% overall, 49↑ 152↓)
Totals:
|
Benchmarks: TPC-H SF=10 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (1.005x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.996x ➖, 1↑ 0↓)
duckdb / vortex-compact / ns (1.028x ➖, 0↑ 1↓)
duckdb / parquet / ns (0.995x ➖, 0↑ 0↓)
File Size Changes (9 files changed, -56.0% overall, 0↑ 9↓)
Totals:
|
Benchmarks: TPC-H SF=1 on S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (1.059x ➖, 0↑ 1↓)
datafusion / parquet / ns (1.027x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (1.042x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.044x ➖, 0↑ 0↓)
|
Benchmarks: Appian on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (1.005x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.000x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (0.995x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.004x ➖, 0↑ 0↓)
File Size Changes (10 files changed, -63.8% overall, 0↑ 10↓)
Totals:
|
Benchmarks: FineWeb S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (1.220x ➖, 0↑ 2↓)
datafusion / parquet / ns (1.180x ➖, 0↑ 1↓)
duckdb / vortex-compact / ns (1.023x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.828x ➖, 1↑ 0↓)
|
Benchmarks: TPC-H SF=1 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.000x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.996x ➖, 1↑ 2↓)
duckdb / vortex-file-compressed / ns (1.009x ➖, 0↑ 1↓)
duckdb / parquet / ns (0.999x ➖, 0↑ 0↓)
File Size Changes (9 files changed, -43.9% overall, 0↑ 9↓)
Totals:
|
Benchmarks: Clickbench on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (1.006x ➖, 1↑ 2↓)
datafusion / parquet / ns (1.003x ➖, 0↑ 1↓)
duckdb / vortex-compact / ns (1.004x ➖, 0↑ 2↓)
duckdb / parquet / ns (1.001x ➖, 1↑ 0↓)
File Size Changes (101 files changed, -60.8% overall, 0↑ 101↓)
Totals:
|
Benchmarks: Statistical and Population Genetics 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
duckdb / vortex-compact / ns (0.989x ➖, 1↑ 1↓)
duckdb / parquet / ns (0.994x ➖, 0↑ 0↓)
File Size Changes (2 files changed, -67.7% overall, 0↑ 2↓)
Totals:
|
Benchmarks: Statistical and Population Genetics 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
duckdb / vortex-file-compressed / ns (0.986x ➖, 2↑ 2↓)
duckdb / parquet / ns (0.992x ➖, 0↑ 0↓)
File Size Changes (2 files changed, -32.3% overall, 0↑ 2↓)
Totals:
|
Benchmarks: FineWeb S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.116x ➖, 1↑ 3↓)
datafusion / parquet / ns (1.003x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.033x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.623x ✅, 2↑ 0↓)
|
Benchmarks: TPC-H SF=10 on S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-compact / ns (1.054x ➖, 0↑ 1↓)
datafusion / parquet / ns (1.004x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (0.995x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.032x ➖, 0↑ 1↓)
|
Benchmarks: FineWeb NVMe 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.999x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.975x ➖, 1↑ 0↓)
duckdb / vortex-file-compressed / ns (0.944x ➖, 5↑ 1↓)
duckdb / parquet / ns (1.025x ➖, 0↑ 0↓)
File Size Changes (2 files changed, -46.3% overall, 0↑ 2↓)
Totals:
|
Benchmarks: TPC-H SF=10 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.003x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.994x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.001x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.984x ➖, 1↑ 0↓)
File Size Changes (9 files changed, -44.0% overall, 0↑ 9↓)
Totals:
|
Benchmarks: TPC-H SF=1 on S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.068x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.024x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed / ns (1.015x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.044x ➖, 0↑ 0↓)
|
Benchmarks: PolarSignals Profiling 📖Vortex (geomean): 1.008x ➖ datafusion / vortex-file-compressed / ns (1.008x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-DS SF=1 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.001x ➖, 1↑ 1↓)
datafusion / parquet / ns (0.997x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.996x ➖, 7↑ 7↓)
duckdb / parquet / ns (1.000x ➖, 5↑ 3↓)
File Size Changes (25 files changed, -43.5% overall, 0↑ 25↓)
Totals:
|
Benchmarks: Clickbench on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.996x ➖, 2↑ 2↓)
datafusion / parquet / ns (0.993x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.030x ➖, 0↑ 8↓)
duckdb / parquet / ns (1.010x ➖, 0↑ 1↓)
File Size Changes (101 files changed, -39.2% overall, 0↑ 101↓)
Totals:
|
Benchmarks: Clickbench Sorted on NVME 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.007x ➖, 1↑ 1↓)
datafusion / parquet / ns (0.986x ➖, 1↑ 0↓)
duckdb / vortex-file-compressed / ns (0.954x ➖, 1↑ 0↓)
duckdb / parquet / ns (1.001x ➖, 0↑ 1↓)
File Size Changes (201 files changed, -42.8% overall, 50↑ 151↓)
Totals:
|
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
32ad0bf to
b490007
Compare
Tracking Issue: #9130
Moves primitive numeric arithmetic and comparisons to
RowFn. Selected measured comparison cases retain their columnar paths.TODO