简体中文 | English
Apache Iceberg is a new open-source table format used for large-scale data analysis. It is designed to store large, slow-moving table data. It aims to improve the de facto standard table layout built into Hive, Trino (PrestoSQL), and Spark. Iceberg can mask differences in underlying data storage formats and provide a unified API for operations, allowing different engines to access through its provided API.
Core Features:
- Schema evolution: Supports adding, dropping, updating, renaming, and reordering table format definitions.
- Partition layout evolution: The layout of a table can be updated with changes in data volume or query patterns.
- Hidden partitioning: The query no longer depends on the physical layout of the table. By separating physical and logical data, Iceberg tables can develop partitioning schemes as the amount of data changes and time passes. Tables with incorrect configurations can be fixed without the need for expensive migration.
- Time travel: supports users to use identical snapshots for repeated queries, or allows users to easily check for changes.
- Version rollback: allows users to quickly correct issues by resetting tables to a good state.
Architecture Design:
This project offers pre-configured Iceberg data storage format images with Iceberg and its runtime environment pre-installed, along with deployment templates. Follow the guide to enjoy an "out-of-the-box" experience.
System Requirements:
- CPU: 2GHz or higher
- RAM: 4GB or more
- Disk: At least 40GB
Register a Huawei account and activate Huawei Cloud
| Image Version | Description | Notes |
|---|---|---|
| Iceberg1.6-kunpeng-v1.0 | Deployed on Kunpeng servers with Huawei Cloud EulerOS 2.0 64bit | |
| Iceberg1.6-kunpeng-v1.0 | Deployed on Kunpeng servers with Ubuntu24.04 64bit |
- Submit an issue
- Contact Huawei Cloud Marketplace product support
- Fork this repository and submit a merge request.
- Update README.md synchronously based on your open-source mirror information.