-
Notifications
You must be signed in to change notification settings - Fork 997
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
One of the goals of adding Variant
support to the parquet crate is to not requiring users to compile and bloat their binary size unless they want this functionality. This means that we have added all actual code and functionality in different crates: parquet_variant
, parquet_variant_compute
, etc.
However at the moment, users who want to use the Variant
functionality need to add the parquet_variant
crate to their Cargo.toml
. This has several downsides:
-
It will be more complicated to use the
Variant
functionality for users -
There is no way to provide end to end examples of reading and writing
Variant
data in either
theparquet
orparquet_variant
crates -
The end to end tests for reading parquet files with
Variant
data need to add both
theparquet
andparquet_variant
crates to theirCargo.toml
Describe the solution you'd like
I would like an easy way to for users to find the Variant feature, and see examples of how to use it
Describe alternatives you've considered
Thus, I propose we add a variant
feature to the parquet
crate that re-exports the parquet_variant
and parquet_variant_compute
crates. This way users can add the parquet
crate with the variant
feature to their Cargo.toml
and get all the functionality.
This is similar to the arrow
feature in the parquet
crate that enables arrow functionality for those users who want it but does not require all users to depend on the arrow
crate.
An alternate strategy would be to require users to explicitly import the
parquet
and parquet_variant
crates in their Cargo.toml
to use the variant
functionality. You can see an example of how this might look in
Additional context
This first came up in the context of adding an integration test for the parquet files here: