Cannot read encrypted Parquet file if page index reading is enabled

[adamreeve]

Describe the bug

Trying to read a Parquet file that uses modular encryption when page indices are enabled in the ArrowReaderOptions results in an error like:

ArrowError("Parquet argument error: External: bad data")

To Reproduce

This test reproduces the issue when added to parquet/tests/encryption/encryption_async.rs:

#[tokio::test]
async fn test_read_with_page_index() {
    let test_data = arrow::util::test_util::parquet_test_data();
    let path = format!("{test_data}/uniform_encryption.parquet.encrypted");
    let mut file = File::open(&path).await.unwrap();

    let key_code: &[u8] = "0123456789012345".as_bytes();
    let decryption_properties = FileDecryptionProperties::builder(key_code.to_vec())
        .build()
        .unwrap();

    let options = ArrowReaderOptions::new()
        .with_file_decryption_properties(decryption_properties)
        .with_page_index(true);

    let arrow_metadata = ArrowReaderMetadata::load_async(&mut file, options)
        .await
        .unwrap();

    let record_reader = ParquetRecordBatchStreamBuilder::new_with_metadata(
        file,
        arrow_metadata,
    )
    .build()
    .unwrap();
    let _record_batches = record_reader.try_collect::<Vec<_>>().await.unwrap();
}

Expected behavior
Data should be read successfully, and give the same results as when with_page_index(false) is used.

Additional context

This was encountered by @corwinjoy when integrating encryption support in DataFusion. Page indexes are enabled when data is queried with a filter predicate.

[adamreeve]

Looking at this more closely, the problem is in SerializedPageReader::get_next_page, which doesn't handle encrypted pages when self.state is SerializedPageReaderState::Pages. An error is encountered when reading the page header here:

arrow-rs/parquet/src/file/serialized_reader.rs

Line 884 in 6a6c631

let header = PageHeader::read_from_in_protocol(&mut prot)?;

Looking at the surrounding code, SerializedPageReaderState::peek_next_page and SerializedPageReaderState::skip_next_page might also need to be updated to work with encrypted pages.