Skip to content

manuzhang/iceberg-examples

Repository files navigation

Iceberg Examples

A collection of examples demonstrating the Apache Iceberg API across multiple languages, built and tested from a single Bazel workspace.

Build and Test

Use Bazel at the repository root for every sub-project:

bazel build //...
bazel test //...

The default Java toolchain is JDK 17. JDK 21 is also supported through an explicit Bazel config:

bazel build --config=jdk21 //...
bazel test --config=jdk21 //...

Common runnable targets:

bazel run //iceberg-java:iceberg_examples
bazel run //iceberg-java:data_operations_example
bazel run //iceberg-java:schema_evolution_example
bazel run //iceberg-java:table_format_v3_example
bazel run //iceberg-java:beam_iceberg_example
bazel run //iceberg-rust:iceberg_rust_examples
bazel run //iceberg-cpp:iceberg_cpp_example

The Spark examples are validated through Bazel as Python modules:

bazel test //iceberg-spark:spark_examples_smoke_test
bazel test //iceberg-flink:flink_sql_smoke_test

Running the Spark pipelines themselves still requires a Spark 4.1 environment plus an Iceberg runtime JAR. See iceberg-spark/README.md for the runtime-specific steps.

Sub-projects

Directory Language Description
iceberg-java/ Java Examples using the Apache Iceberg Java API, including Apache Beam
iceberg-rust/ Rust Examples using the Apache Iceberg Rust API
iceberg-spark/ Python / SQL Spark 4.1 Declarative Pipelines with Apache Iceberg
iceberg-flink/ SQL Flink SQL Iceberg catalog and table workflow example
iceberg-cpp/ C++ Minimal Iceberg-style schema modeling example in C++

Repository Layout

  • MODULE.bazel defines the shared Maven, Rust crate, and PyPI dependencies.
  • iceberg-java/BUILD.bazel exposes Java libraries, runnable examples, and tests.
  • iceberg-rust/BUILD.bazel exposes the Rust example binary and unit tests.
  • iceberg-spark/BUILD.bazel validates the Spark Python examples and exports the SQL pipeline.
  • iceberg-flink/BUILD.bazel exports and validates the Flink SQL Iceberg example script.
  • iceberg-cpp/BUILD.bazel builds a runnable C++ Iceberg-style schema example.

Learning Resources

License

This project is licensed under the same terms as the Apache Iceberg project.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors