Skip to content

Make Sbbf Constructers Public#9569

Merged
alamb merged 1 commit intoapache:mainfrom
pydantic:sbbf_public_methods
Mar 20, 2026
Merged

Make Sbbf Constructers Public#9569
alamb merged 1 commit intoapache:mainfrom
pydantic:sbbf_public_methods

Conversation

@cetra3
Copy link
Copy Markdown
Contributor

@cetra3 cetra3 commented Mar 18, 2026

Which issue does this PR close?

None

Rationale for this change

We want to use the SBBF Bloom Filter, but need to construct/serialize it manually. Currently there is no way to create a new Sbbf outside of this crate. Alongside this: we want to store the Sbbf in a FixedSizedBinary column for some fancy indexing.

What changes are included in this PR?

Some methods become public

Are these changes tested?

N/A

Are there any user-facing changes?

Yes, we add a few more public methods to the Sbbf struct

@github-actions github-actions Bot added the parquet Changes to the parquet crate label Mar 18, 2026
@cetra3 cetra3 force-pushed the sbbf_public_methods branch from 91c414d to dc8aba8 Compare March 18, 2026 07:12
@cetra3 cetra3 force-pushed the sbbf_public_methods branch from dc8aba8 to d003d88 Compare March 19, 2026 00:09
Copy link
Copy Markdown
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me -- thank you @cetra3

BTW it might be good to add some public docs / tests to show more how this is ued

@alamb alamb merged commit 39dda22 into apache:main Mar 20, 2026
16 checks passed
friendlymatthew pushed a commit to pydantic/arrow-rs that referenced this pull request Mar 21, 2026
# Which issue does this PR close?

None

# Rationale for this change

We want to use the SBBF Bloom Filter, but need to construct/serialize it
manually. Currently there is no way to create a new `Sbbf` outside of
this crate. Alongside this: we want to store the `Sbbf` in a
`FixedSizedBinary` column for some fancy indexing.

# What changes are included in this PR?

Some methods become public

# Are these changes tested?

N/A

# Are there any user-facing changes?

Yes, we add a few more public methods to the `Sbbf` struct
marc-pydantic added a commit to pydantic/datafusion-distributed that referenced this pull request Apr 28, 2026
Point dependencies to pydantic forks which include:

arrow-rs (pydantic-main @ 58.0.0):
- sparse data fix: apache/arrow-rs#9509
- public Sbbf methods: apache/arrow-rs#9569
- string view unshred variant: apache/arrow-rs#9514
- fix extend_nulls panic for UnionArray: apache/arrow-rs#9607

datafusion (pydantic-main @ b4116da0d):
- union coercion fixes: apache/datafusion#18825
- GC spilling fix: apache/datafusion#19444
- add arrow_try_cast UDF: apache/datafusion#21130
- leaf-level ProjectionMask for parquet projections: apache/datafusion#20925
- parquet Variant UDF filter pushdown with leaf pruning
- support union comparison for try-cast
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parquet Changes to the parquet crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants