Describe the bug
We created a StreamReader with with_skip_validation(true) and still ran into validation errors.
We noticed that not all validation steps are skipped. The record_batch_reader
function does not respect the skip_validation property and simply calls RecordBatch::try_new_with_options instead of RecordBatch::new_unchecked.
To Reproduce
Expected behavior
Skip the validation logic by calling RecordBatch::new_unchecked if skip_validation is set to true.
Additional context
Describe the bug
We created a
StreamReaderwithwith_skip_validation(true)and still ran into validation errors.We noticed that not all validation steps are skipped. The record_batch_reader
function does not respect the
skip_validationproperty and simply callsRecordBatch::try_new_with_optionsinstead ofRecordBatch::new_unchecked.To Reproduce
Expected behavior
Skip the validation logic by calling
RecordBatch::new_uncheckedifskip_validationis set to true.Additional context