Skip to content

[Variant] Add variant feature to parquet crate #8132

@alamb

Description

@alamb

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
One of the goals of adding Variant support to the parquet crate is to not requiring users to compile and bloat their binary size unless they want this functionality. This means that we have added all actual code and functionality in different crates: parquet_variant, parquet_variant_compute, etc.

However at the moment, users who want to use the Variant functionality need to add the parquet_variant crate to their Cargo.toml. This has several downsides:

  1. It will be more complicated to use the Variant functionality for users

  2. There is no way to provide end to end examples of reading and writing Variant data in either
    the parquet or parquet_variant crates

  3. The end to end tests for reading parquet files with Variant data need to add both
    the parquet and parquet_variant crates to their Cargo.toml

Describe the solution you'd like

I would like an easy way to for users to find the Variant feature, and see examples of how to use it

Describe alternatives you've considered
Thus, I propose we add a variant feature to the parquet crate that re-exports the parquet_variant and parquet_variant_compute crates. This way users can add the parquet crate with the variant feature to their Cargo.toml and get all the functionality.

This is similar to the arrow feature in the parquet crate that enables arrow functionality for those users who want it but does not require all users to depend on the arrow crate.

An alternate strategy would be to require users to explicitly import the
parquet and parquet_variant crates in their Cargo.toml to use the variant
functionality. You can see an example of how this might look in

Additional context
This first came up in the context of adding an integration test for the parquet files here:

Metadata

Metadata

Assignees

Labels

enhancementAny new improvement worthy of a entry in the changelog

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions