[Arrow] Add bitwise operations `BooleanArray` that potentially reuse the underlying allocation

[alamb]

This is related to

• Consolidate bitwise operation implementations #8806
• Hard to use PrimitiveArray::unary_mut, PrimitiveArray:try_unary_mut, etc #8808

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

It is common to apply successive filtering operations that progressively narrow a bitmap with rows that pass a predicate -- for example a AND b AND c AND d AND ...

This would be evaluated typically like

t1 = a AND b
t2 = t1 AND c
t3 = t2 AND d

Where t1, t2, and t3 are newly allocated arrays; t3 holds the final output

For a typical array size of 8192 elements, t1, t2 and t3 are eack 1KB allocations

The current boolean kernels in arrow pretty much require allocating a new buffer as they take a reference

Now that @rluvaton has added optimized code to apply mutable bitwise operations in the following PR

• feat: add apply_unary_op and apply_binary_op bitwise operations #8619

I think we are in the position to also provide the same operations on BooleanArray

Describe the solution you'd like

I would like a way to update the contents of a BooleanArray in place (assuming there is no other reference to the backing array)

Describe alternatives you've considered
I propose following the API of PrimitiveArray, such as PrimitiveArray::unary and PrimitiveArray::unary_mut

So that would mean adding new functions to BooleanArray, like

impl BooleanArray {
...
  // apply a bitwise operation to this BooleanArray, using u64 operations
  // returning a new BooleanArray
  fn bitwise_unary(&self, mut op:F) -> Self
where
    F: FnMut(u64) -> u64 {.. }

  // Try to apply a bitwise operation to this BooleanArray in place, using u64 operations
  // If the underlying buffer is shared, returns Err<self>
  fn bitwise_unary_mut(self, mut op:F) -> Result<Self, Self>
where
    F: FnMut(u64) -> u64 {.. }

  // Try to apply a bitwise operation to this BooleanArray in place, using u64 operations
  // If the underlying buffer is shared, returns a new 
  // BooleanArray
  fn bitwise_unary_mut_or_clone(self, mut op:F) -> Self
where
    F: FnMut(u64) -> u64 {.. }

And we would also add similar functions for binary_bitwise functions (that takes a second array argument and right_offset_in_bits)

These APIs are likely fairly straightforward if we implement the plan in

• Consolidate bitwise operation implementations #8806

As then the new methods on BooleanArray would just call the relevant underlying buffer methods

Additional context

[rluvaton]

I think the title is misleading, there can't be mutation on Arrow Array due to the nature of immutable arrays in arrow

[alamb]

I think the title is misleading, there can't be mutation on Arrow Array due to the nature of immutable arrays in arrow

Good call -- I tried to update it to better reflect the idea

[alamb]

@Dandandan just added in place bitwise operations:

• Add mutable operations to BooleanBuffer (Bit*Assign) #9567