Skip to content

octet_length should accept binary types #24765

Description

@eddietejeda

Is your feature request related to a problem or challenge?

octet_length only accepts string types. When we use a binary column, the plan fails:

Function 'octet_length' requires String, but received Binary (DataType: BinaryView)

A workaround for SUM(octet_length(col)) over binary is:

SUM(length(encode(col, 'hex')) / 2)

But this materializes 2x the column's bytes just to count it.

Describe the solution you'd like

Extend octet_length to natively accept Binary, LargeBinary, and BinaryView, returning the byte length.

  • SQL:1999 defines OCTET_LENGTH for string values, including binary strings.
  • PostgreSQL, DuckDB, Spark, MySQL, and SQLite accept binary values.
  • utf8_to_int_type already maps Binary -> Int32, LargeBinary -> Int64, and BinaryView -> Int32.

We've implemented the solution internally and happy to submit PR.

datafusion/functions/src/string/octet_length.rs

Before:

impl OctetLengthFunc {
    pub fn new() -> Self {
        Self {
            signature: Signature::coercible(
                vec![
                    Coercion::new_exact(TypeSignatureClass::Native(logical_string()))
                        .with_encoding_preservation(EncodingPreservation::dictionary()),
                ],
                Volatility::Immutable,
            ),
        }
    }
}

After: the two-branch string|binary select that md5 uses:

impl OctetLengthFunc {
    pub fn new() -> Self {
        Self {
            signature: Signature::one_of(
                vec![
                    TypeSignature::Coercible(vec![
                        Coercion::new_exact(TypeSignatureClass::Native(logical_string()))
                            .with_encoding_preservation(
                                EncodingPreservation::dictionary(),
                            ),
                    ]),
                    TypeSignature::Coercible(vec![
                        Coercion::new_exact(TypeSignatureClass::Native(logical_binary()))
                            .with_encoding_preservation(
                                EncodingPreservation::dictionary(),
                            ),
                    ]),
                ],
                Volatility::Immutable,
            ),
        }
    }
}

Describe alternatives you've considered

 length(encode(col, 'hex')) / 2 // works but doubles the data to count it

Additional context

bit_length has the same gap and could be part in the same PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions