Test `thrift-remodel` branch with DataFusion

[alamb]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

• part of Use custom thrift decoder to improve speed of parsing parquet metadata #5854

Describe the solution you'd like
The idea is to do some early testing with downstream projects to ensure that the new thrift decoding can be used downstream / has no hidden surprises.

Since I am quite familiar with DataFusion, I would like to test with that project

Describe alternatives you've considered

Additional context

[alamb]

My plan here is twofold:

1. A WP PR to upgrade to arrow 57.0.0 (main of arrow-rs) so I can work out any issues unrelated to thrift-upgrade
2. A second PR (based on the first) to actually test the thrift remodel

[alamb]

PR to upgrade datafusion to latest arrow main:

• Upgrade DataFusion to arrow/parquet 57.0.0 datafusion#17888

[alamb]

Ed has a branch to test against here:

• Use custom thrift parser for parquet metadata (phase 1 of Thrift remodel) #8530

[alamb]

I made a PR to start testing here:

• [TESTING] Test thrift-remodel branch with datafusion datafusion#17916

I am still working through the API changes , and it doesn't yet compile

[alamb]

Update, functionally all tests are passing. I have scheduled some benchmarks and will report on the status later today

[alamb]

The benchmarks look good to me. In my opinion the thrift-remodel branch is ready to go 🚀

[alamb]

label_issue.py automatically added labels {'parquet'} from #8476