Is your feature request related to a problem or challenge? Please describe what you are trying to do.
|
/// Returns an array of Int32/Int64 denoting the number of bits in each value in the array. |
|
/// |
|
/// * this only accepts StringArray/Utf8, LargeString/LargeUtf8, BinaryArray and LargeBinaryArray, |
|
/// or DictionaryArray with above Arrays as values |
|
/// * bit_length of null is null. |
|
/// * bit_length is in number of bits |
|
pub fn bit_length(array: &dyn Array) -> Result<ArrayRef, ArrowError> { |
|
if let Some(d) = array.as_any_dictionary_opt() { |
|
let lengths = bit_length(d.values().as_ref())?; |
|
return Ok(d.with_values(lengths)); |
|
} |
|
|
|
match array.data_type() { |
|
DataType::List(_) => { |
|
let list = array.as_list::<i32>(); |
|
Ok(bit_length_impl::<Int32Type>(list.offsets(), list.nulls())) |
|
} |
|
DataType::LargeList(_) => { |
|
let list = array.as_list::<i64>(); |
|
Ok(bit_length_impl::<Int64Type>(list.offsets(), list.nulls())) |
|
} |
The bit_length kernel documentation states it is only for strings, and the way it calculates bit length for lists assumes each element in a list = 1 byte. I think this may have been a mistake, and we should remove this handling for clarity.
Describe the solution you'd like
Remove support for List types in bit_length kernel.
Describe alternatives you've considered
Keep this behaviour since it's been like this for a while now and might be considered breaking behaviour change
Additional context
Seems it was introduced by
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
arrow-rs/arrow-string/src/length.rs
Lines 111 to 131 in 7291147
The
bit_lengthkernel documentation states it is only for strings, and the way it calculates bit length for lists assumes each element in a list = 1 byte. I think this may have been a mistake, and we should remove this handling for clarity.Describe the solution you'd like
Remove support for
Listtypes inbit_lengthkernel.Describe alternatives you've considered
Keep this behaviour since it's been like this for a while now and might be considered breaking behaviour change
Additional context
Seems it was introduced by