Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Often we would like to display DataTypes as part of an error message in DataFusion
The documentation (link) says
The Display and FromStr implementations for DataType are human-readable, parseable, and reversible.
However this is not the case for ListArray
In apache/datafusion#14378 I made what looks like it should be a nice error message
format!("It is not possible to concatenate arrays of different types. Expected: {}, got: {}", expr_type, arg_type)
However, what comes out is is 🤮
query error DataFusion error: Error during planning: It is not possible to concatenate arrays of different types. Expected: List(Field { name: "item", data_type: Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }), got: List(Field { name: "item", data_type: LargeUtf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} })
Describe the solution you'd like
I would like a way to get a nicer human readable version of this for error messages
Soemthing like
Expected: List(Utf8;N), got: List(LargeUtf8;N)
Where the N stands for nullable
Describe alternatives you've considered
I personally recommend
- Explicitly implemeting
Display for DataType::List
- Don't print out any information that is the default (e.g if the field name is "item" then don't print it)
Right now it simply uses the Debug implementation:
|
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { |
|
write!(f, "{self:?}") |
|
} |
Additional context
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Often we would like to display DataTypes as part of an error message in DataFusion
The documentation (link) says
However this is not the case for ListArray
In apache/datafusion#14378 I made what looks like it should be a nice error message
However, what comes out is is 🤮
Describe the solution you'd like
I would like a way to get a nicer human readable version of this for error messages
Soemthing like
Where the
Nstands for nullableDescribe alternatives you've considered
I personally recommend
DisplayforDataType::ListRight now it simply uses the
Debugimplementation:arrow-rs/arrow-schema/src/datatype.rs
Lines 460 to 462 in 3bf29a2
Additional context