Question about type conversion between parquet and arrow.

[RinChanNOWWW]

parquet_to_arrow_schema converts Type::Int96 to DataType::Timstamp(TimeUnit::Nano).

However, arrow_to_parquet_schema converts DataType::Timstamp(TimeUnit::Nano) to Type::Int96.

Why?

[tustvold]

Int96 is deprecated - https://github.com/apache/parquet-format/blob/master/Encodings.md#plain-plain--0

Timestamps should always be encoded as int64 now - https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#timestamp

Int96 is only still supported for compatibility with spark, which still has issues supporting nanosecond timestamps stored as int64