Skip to content

Remove support for List types in bit_length kernel #9349

@Jefffrey

Description

@Jefffrey

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

/// Returns an array of Int32/Int64 denoting the number of bits in each value in the array.
///
/// * this only accepts StringArray/Utf8, LargeString/LargeUtf8, BinaryArray and LargeBinaryArray,
/// or DictionaryArray with above Arrays as values
/// * bit_length of null is null.
/// * bit_length is in number of bits
pub fn bit_length(array: &dyn Array) -> Result<ArrayRef, ArrowError> {
if let Some(d) = array.as_any_dictionary_opt() {
let lengths = bit_length(d.values().as_ref())?;
return Ok(d.with_values(lengths));
}
match array.data_type() {
DataType::List(_) => {
let list = array.as_list::<i32>();
Ok(bit_length_impl::<Int32Type>(list.offsets(), list.nulls()))
}
DataType::LargeList(_) => {
let list = array.as_list::<i64>();
Ok(bit_length_impl::<Int64Type>(list.offsets(), list.nulls()))
}

The bit_length kernel documentation states it is only for strings, and the way it calculates bit length for lists assumes each element in a list = 1 byte. I think this may have been a mistake, and we should remove this handling for clarity.

Describe the solution you'd like

Remove support for List types in bit_length kernel.

Describe alternatives you've considered

Keep this behaviour since it's been like this for a while now and might be considered breaking behaviour change

Additional context

Seems it was introduced by

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementAny new improvement worthy of a entry in the changelog

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions