Implementation of LowPass, HighPass, and XCorr Table Functions

[alpass163gmail]

LowPass and HighPass

The LowPass and HighPass functions accept the following parameters:

• DATA: the input table.
• TIMECOL: the time column used for ordering. The ORDER BY clause must contain only this time column.
• WPASS: a double literal in the range (0, 1), which defines the frequency boundary.

For each calculation column, the function collects the complete sequence of finite, non-null values and applies an FFT-based transformation:

• LowPass keeps the low-frequency components and removes the high-frequency components.
• HighPass removes the low-frequency components and keeps the high-frequency components.

The inverse FFT is then applied to obtain the transformed sequence. The output contains the partition columns, the original time column, and one DOUBLE result column for each calculation column. Null, NaN, and infinite input values are not included in the FFT calculation.

XCorr

The XCorr function accepts:

• DATA: the input table.
• TIMECOL: the time column used for ordering. The ORDER BY clause must contain only this time column.
• PARTITION BY: optional partition columns.
• Exactly two calculation columns after excluding the partition columns and the time column.

The two calculation columns are treated as two aligned sequences. The function calculates their cross-correlation for all lags from -(n - 1) to n - 1, where n is the number of rows in the partition.

For each lag, only pairs in which both values are finite and non-null participate in the calculation. The correlation value is normalized by the number of valid pairs for that lag. If a lag has no valid pairs, the corresponding result is returned as null.

The output contains the partition columns and one DOUBLE result column named according to the two calculation columns, for example xcorr(s1, s2). The original time column is used for ordering but is not included in the result.