Apache Iceberg vs Apache Hudi

Choose Iceberg when many engines must share tables and update volume is moderate. Choose Hudi when record-level upserts, deletes and CDC drive the design. Both are Apache-2.0 and self-hosted.

On this page

Apache Iceberg vs Apache Hudi: when each one fits

A team choosing a table format for upserts, deletes and change data capture usually wants to know which format will serve its engines and its operators for the next several years. Apache Iceberg fits best when the same tables must be shared by several engines at once. The format is described by its project as a high-performance format for huge analytic tables that lets engines such as Spark, Trino, Flink, Presto, Hive and Impala work with the same tables at the same time. Iceberg also lists schema evolution without rewriting tables, hidden partitioning, time travel to exact historical snapshots and rollback to an earlier table state as core features. If your priority is broad engine access, a clean governance path for schema changes and the ability to audit or roll back table history, Iceberg is the natural starting point, and it is licensed under the Apache License 2.0.

Apache Hudi fits best when record-level change handling is the center of the workload. Hudi describes itself as an open data lakehouse platform and ships built-in ingestion tools for Spark and Flink users plus a Kafka Connect sink for external data. Its record-level indexes support fast upserts and deletes, and it can serve snapshot, incremental and change-data-capture queries on one table, with before and after images. Its Merge On Read tables write updates to log files that are merged with base files at query time and folded into new base files by compaction, which keeps write latency low for high-churn tables. The trade-off is that read-optimized queries only see data up to the last compaction, so you must confirm that this freshness window is acceptable for your consumers. Hudi is also Apache-2.0 licensed.

Four decision drivers should settle the choice. First, list every engine that must read and write the table, because Iceberg's multi-engine reach is its strongest argument and you should check engine support for each version you run. Second, estimate delete volume and the latency your downstream users need, since high-frequency row-level changes favor Hudi's indexes and Merge On Read layout. Third, decide who will operate compaction, cleaning and clustering, and whether those services will run inline with writers or as separate jobs; a team without streaming operations experience should weigh that effort heavily. Fourth, ask your platform and security teams how each format's catalog, access control and audit trail will fit your existing estate, and confirm those details against the current Apache Iceberg documentation and the Hudi table types documentation before committing.

Which scenarios favor Apache Iceberg and which favor Apache Hudi

This table maps common workloads to the format with the stronger fit, based on each project's documented write paths, table types and engine support. Read the last column as the condition under which the other format becomes the better choice. The main takeaway is that Hudi leads when record-level change handling drives the design, while Iceberg leads when shared multi-engine reads and table history matter most.

ScenarioBetter fitWhy, from the documented featuresChoose the other platform when
High-frequency CDC from operational databasesApache HudiRecord-level indexes and Merge On Read log files absorb frequent updates and deletes, and CDC reads include before and after images.Change volume is modest and one table format shared across engines matters more than per-record upsert speed. Choose Iceberg.
Mostly append analytics with occasional correctionsApache IcebergSchema evolution, hidden partitioning and snapshot rollback suit append-heavy tables, and v2 delete files handle occasional corrections.Corrections arrive continuously and need record-level indexing and tuned compaction. Choose Hudi.
Multi-engine reads across Spark, Trino and FlinkApache IcebergIceberg is designed for Spark, Trino, Flink, Presto, Hive and Impala to work on the same tables at the same time.One Spark or Flink pipeline owns both writes and reads. Confirm reader support for each engine before choosing Hudi.
Streaming writes with late dataApache HudiNon-blocking concurrency control targets streaming writes with late data, and built-in Spark and Flink ingestion tools are included.Writers are few, late data is rare and a standard append path is enough. Choose Iceberg.
Read-heavy tables with infrequent writesApache Hudi (Copy On Write)Copy On Write writes new base files with no log files, giving high read performance at the cost of slower writes.Time travel and rollback to exact snapshots matter more than read speed alone. Choose Iceberg.
Audit history and point-in-time investigationApache IcebergTime travel to exact snapshots and rollback are core features, and Athena engine version 3 supports both time travel clauses.Commit-level rollback, restore and incremental queries drive the design. Choose Hudi.
Managed cloud platform interoperabilityApache IcebergSnowflake and OneLake both expose Iceberg tables to external readers, and Trino reads Iceberg through several catalog types.Your target platform reads Hudi tables. Confirm this with the vendor before design work starts.
Ownership of table maintenance jobsApache HudiCleaning, clustering and compaction can run inline with Spark or Flink writers or as independent services.Your team wants maintenance scheduled outside the writer. Choose Iceberg and confirm which maintenance jobs your engines run.

How Hudi's record indexes and Iceberg's metadata tree handle upserts

Upserts and deletes stress four parts of a lakehouse table: how writes land, how the engine finds the rows to change, how maintenance keeps reads fast, and how concurrent commits stay consistent. The cards below take each in turn and name what your team would run, monitor and confirm before choosing between the two formats.

Write path and read cost

Hudi Copy On Write tables create new base files for each update and delete, which favors reads but slows writes. Merge On Read tables append updates to log files that are merged with base files at query time, and compaction later folds them into new base files. Iceberg v2 delete files and v3 deletion vectors follow a different merge pattern, so confirm which delete mechanism your writer produces and how your query engine applies it. Test write latency and read freshness at your real upsert rate.

Locating the rows to change

Hudi uses record-level indexes to find existing records quickly during upserts and deletes, which is the main reason it is built for high-volume change streams. The supplied material does not describe a comparable record-level index in Iceberg, so do not assume one. Ask the Iceberg writer and engine you plan to use how an update identifies its target rows, and benchmark that path with your own key cardinality and partitioning before committing to a design.

Table service load and read freshness

Hudi cleaning, clustering and compaction can run inline inside Spark or Flink writers or as independent services. Inline operation keeps fewer moving parts but ties maintenance to writer jobs, while separate services need their own scheduling and monitoring. Merge On Read reads see data only up to the last compaction, so compaction cadence sets how fresh read-optimized queries are. For Iceberg, confirm with your engine which snapshot expiration and file cleanup procedures you must schedule and who owns them.

Catalog and commit model

Hudi commits atomically, supports rollback and restore, and offers optimistic concurrency control plus non-blocking concurrency control for streaming writes that receive late data. Iceberg organizes each table state as a snapshot, supports time travel to exact historical snapshots and rollback to an earlier state, and connects engines through a catalog. In Trino, that catalog is selected with iceberg.catalog.type. Map which engines and writers must share one catalog and which can commit concurrently before you pick either format.

Apache Iceberg vs Apache Hudi on deletes, upserts and CDC feature by feature

This table compares how Apache Iceberg and Apache Hudi handle deletes, upserts, change data capture and concurrency, using only capabilities confirmed in the supplied documentation. Rows marked to confirm depend on engine versions or features you must verify in your own stack. Read each row against the workload you run most often, not as a general ranking.

CapabilityApache IcebergApache Hudi
LicenseApache License 2.0, with Java reference implementation and separate Python, Go, Rust and C++ implementationsApache-2.0, an Apache Software Foundation project
Row-level delete mechanismRow-level deletes are handled through v2 delete files; confirm which delete file behavior your engines apply on readsUpdates and deletes are recorded per record through record-level indexes for fast upserts and deletes
Deletion vectorsDeletion vectors are associated with v3 of the format; confirm v3 support in each engine you plan to runNot a named feature in the supplied facts; confirm how your Hudi version marks deleted records
Table types for updatesOne table layout; update behavior depends on the delete files and engine write path you configureCopy On Write rewrites base files with no log files; Merge On Read writes log files merged at query time
Compaction of pending changesConfirm the compaction or rewrite jobs your engine and catalog expect for accumulated delete filesMerge On Read compaction periodically folds log files into new base files; read-optimized queries see data only up to the last compaction
Change data captureNot listed as a core feature in the supplied facts; confirm CDC pipelines against your engine and connectorSupports change-data-capture queries with before and after images on one table
Time travel and rollbackTime travel to exact historical snapshots and rollback to an earlier table state are core featuresTime-travel queries and rollback and restore on committed writes are supported
Concurrency controlConfirm the commit and conflict behavior of your catalog and writer enginesOptimistic concurrency control, plus non-blocking concurrency control for streaming writes with late data
Table service placementMaintenance such as snapshot expiry and file rewrites is typically run by the engine or a separate job; confirm your toolchainCleaning, clustering and compaction can run inline with Spark or Flink writers or be operated independently

Which query engines read Iceberg and Hudi tables

Trino is often the query layer teams choose when they want one SQL engine across several Iceberg catalogs. Its Iceberg connector selects the catalog through the iceberg.catalog.type setting, and the documented options include Hive metastore, AWS Glue, JDBC, REST, Nessie and Snowflake catalogs. The choice of catalog decides how table metadata is stored and shared, so it should be settled before the first table is created.

  • Catalog choice: set iceberg.catalog.type to match the catalog you already operate, or the one your writers will use.
  • File formats: the connector reads Parquet (the default), ORC or Avro data files.
  • Shared tables: the same tables stay readable by other engines, such as Spark or Flink, while Trino serves analysts.
  • Hudi reads: the Iceberg connector documentation does not cover Hudi tables, so a Hudi read path through Trino needs its own confirmation.
  • Delete handling: check how your Trino version handles Iceberg v2 delete files and v3 deletion vectors before you plan merge-heavy workloads.

Confirm the deletion vector and delete file behavior in the Trino Iceberg connector documentation for the exact version you will run.

Moving a Hudi table to Iceberg without losing change history

This sequence moves existing Hudi tables to Apache Iceberg while keeping the change history that downstream users depend on. It ends with Iceberg tables whose reads and deletes have been validated, recent changes replayed, and the Hudi writers retired. Effort depends mainly on delete volume and on which downstream engines must read the new tables.

  1. Inventory every Hudi table and its table type

    The data platform team records each Hudi table, whether it is Copy On Write or Merge On Read, its record key and ordering fields, the indexes it uses, and the writers that feed it, such as Spark, Flink or Kafka Connect. It also lists every downstream reader, scheduled table service, and the approximate volume of deletes per table. The step is done when the owners of each table have signed off on the inventory.

  2. Decide which history must survive

    Owners decide which history the new tables must keep: current state only, the full snapshot trail, or change records with before and after images for CDC consumers. Hudi's commit timeline does not carry over as a set of Iceberg snapshots, so each retained period must be restated in Iceberg terms. Compliance and legal owners should confirm retention requirements in writing. The step is done when each table has a written retention decision.

  3. Choose the catalog and confirm engine support

    The team selects an Iceberg catalog that every reader can reach. Trino's Iceberg connector supports Hive metastore, AWS Glue, JDBC, REST, Nessie and Snowflake catalogs, so the choice should follow the catalog your other engines already use. The team must confirm with each engine vendor which Iceberg format versions it reads, including support for v2 delete files and v3 deletion vectors. The step is done when a test table written in the chosen catalog can be read by every engine on the list.

  4. Create Iceberg tables and backfill the base data

    Engineers map Hudi column types and the record key to an Iceberg schema, then define a partitioning spec, using hidden partitioning where it fits the query patterns. They copy the current state of each table into Iceberg with a batch job, and they reuse existing Parquet base files only after confirming the approach with the Iceberg migration documentation. Row counts and checksums for sampled partitions are compared with the Hudi source. The step is done when the backfill reconciles for every table in scope.

  5. Compact Merge On Read logs before cutover

    For Merge On Read tables, the team runs Hudi compaction so that pending log files fold into base files before the backfill watermark is set. Hudi read-optimized queries only see data up to the last compaction, so an uncompacted table gives an incomplete picture at the point of cutover. Operators confirm that the compaction completed and that no log files remain behind the watermark. The step is done when the watermark is recorded and the source table has no unmerged log files before it.

  6. Replay the change gap and cut over

    After the backfill, the team pauses Hudi writers at an agreed point and replays the change records written after the watermark into Iceberg. Updates and deletes are applied with MERGE statements, which produce v2 delete files, or with deletion vectors where the engines in use support v3. Operators then compare key counts and recent change totals between the two systems until the lag reaches zero. The step is done when the replay reconciles and the Iceberg tables are the write target.

  7. Validate reads, deletes and time travel

    Test queries run on Trino, Spark and any other engine that reads these tables, comparing results with the Hudi source for sampled keys and time ranges. Deleted keys must be absent in every engine, including after snapshot expiry and file maintenance. Time travel is checked against the retention decision from step two, and Amazon Athena's engine version 3 time travel clauses are tested if Athena is a reader. The step is done when the test set passes and its results are signed off by the data owners.

  8. Retire Hudi writers, services and access paths

    The team switches pipelines to write only to Iceberg, disables Hudi cleaning, clustering and compaction jobs, and moves access policies and catalog grants to the Iceberg tables. Hudi tables stay read-only for a retention window that owners agree in writing, with a documented rollback plan. Indexes and pipeline code that only served Hudi are archived or removed. The step is done when no Hudi writer is running and the rollback plan has been reviewed.

What drives the running cost of each format

Running cost for Iceberg and Hudi comes from the same few places: the compute that keeps tables compact, the storage that accumulates between compactions, and the people who tune and upgrade the stack. Use these cards to turn each driver into a line item you can ask a partner or vendor to price against your own workload.

Compaction and table service compute

In Hudi Merge On Read tables, a periodic compaction process folds log files into new base files, and cleaning and clustering run as table services. Iceberg relies on rewrite jobs for file compaction and on snapshot expiration for cleanup. Either way, compute recurs for every hour or day of ingestion, and the schedule you choose sets the bill. Ask any quote to state which services run inline with the writer and which run as separate jobs, and what cluster size each assumes.

Storage growth from logs and snapshots

Merge On Read log files and Iceberg delete files add storage until a compaction or rewrite removes them, and retained snapshots keep older data files alive for time travel and rollback. Storage therefore grows with update rate and with how long you keep history, not only with table size. Ask for the retention policy, the compaction cadence and the expected growth under your update volume, and confirm who sets and enforces those values after handover.

Spark and Flink tuning skills

Both formats depend heavily on how the writer is configured: index choice, compaction triggers, file sizing and concurrency control for Hudi, and delete handling and maintenance procedures for Iceberg. Expert tuning is largely a recurring labor cost, paid through staff time or a services retainer. Ask who on your side will own these settings, which skills the partner will transfer, and whether the quote includes tuning after go-live.

Catalog service operation

Iceberg needs a catalog such as a Hive metastore, AWS Glue, JDBC, REST, Nessie or Snowflake, selected through iceberg.catalog.type in Trino. Hudi uses its own metadata and timeline on the table, so the operational catalog burden differs. Operating a self-hosted catalog adds patching, backup, high availability and access control work that recurs. Ask whether the quote includes catalog hosting, who is paged when it fails, and what backup and restore targets are defined.

Engine licensing and managed fees

Both formats are Apache-2.0 licensed, so the table format itself carries no license fee. Costs appear in the engines and services around it: a managed Spark or Flink service, a managed Trino or query platform, or a cloud catalog and storage bill. Where a platform sells managed table services, those fees are usually recurring and metered. Ask each vendor for its pricing model, which features are gated to paid tiers, and how fees change as tables and writers grow.

Upgrade effort across engine versions

Engine and format upgrades recur, and each one touches writers, readers and maintenance jobs. A table written by one Hudi or Iceberg version must still be readable by every engine that queries it, so mixed fleets need compatibility testing before each upgrade. Plan for regression testing across Spark, Flink and Trino releases on a fixed cadence. Ask for the upgrade plan, the test coverage included, and whether the partner's fee covers upgrades or only the initial build.

Where Iceberg and Hudi implementations usually run long

Hudi projects tend to run longest on tuning that only shows up under real update volume. The decisions that matter are the table type, the index, and the compaction schedule. Merge On Read tables write updates to log files and fold them into base files during compaction, and read-optimized queries only see data up to the last compaction. Teams should load a production-like update rate, measure how far read-optimized results lag, and decide how often compaction must run to keep that lag acceptable. The record-level index that drives fast upserts and deletes also needs sizing against the real key cardinality. Ask the implementer to document the index type, the compaction and cleaning cadence, and the write rate each setting was tested at, and confirm these against the Hudi table types documentation for the Hudi release in use.

Iceberg projects tend to run longest on three decisions made early and hard to reverse. The first is the catalog, since Trino's Iceberg connector alone supports Hive metastore, AWS Glue, JDBC, REST, Nessie and Snowflake catalogs selected through iceberg.catalog.type, and every engine must point at the same one. The second is engine version alignment: Spark, Trino, Flink and any query service must agree on Iceberg support, so the integration test matrix has to cover each pair in use. The third is the delete strategy. Iceberg v2 delete files and v3 deletion vectors change read and maintenance behavior, so teams should test read latency and compaction on realistic delete volumes before committing. Confirm the spec version each engine supports before any table is created.

Across both formats, the same failure points recur. Late-arriving data forces teams to decide how far back a window reopens and how it is reconciled with already published snapshots. Concurrency conflicts appear when batch backfills and streaming writers touch the same table, and Hudi's optimistic and non-blocking concurrency controls need explicit configuration and testing. Multi-engine testing is often underestimated because a write that succeeds in one engine can expose a schema or delete behavior another engine reads differently. Schema evolution rules also need written limits on renames, type changes and partition changes. A fixed-scope engagement should name the specific engines and versions in scope, the write patterns and volumes tested, the compaction and cleaning schedule, the late-data window, the concurrency scenarios, and the schema change policy, with acceptance tests for each.

Licensing, releases and what is paid for each format

Apache Iceberg is licensed under the Apache License 2.0, and its main repository in the apache/iceberg project is the Java reference implementation. Separate implementations exist for Python (PyIceberg), Go, Rust and C++, so teams can read and write Iceberg tables from services that do not run on the JVM. The documentation menu lists 1.12.0 as the latest release, and the Apache Iceberg releases page is the place to confirm the current line before you pin a version.

  • License: Apache License 2.0 applies to the format's reference code, so you can run, modify and redistribute it inside your own infrastructure without a commercial license fee for the library itself.
  • Reference implementation: The Java library in the main repository is the most complete reference for catalogs and engine integrations, and it is the code path most engines build against.
  • Other languages: PyIceberg, Go, Rust and C++ implementations exist, but confirm feature parity for the operations you need (such as row-level deletes or specific catalog types) before relying on them.
  • Documented release line: 1.12.0 is the latest release listed in the documentation menu. Engine connectors, such as Trino's and Spark's, have their own version compatibility, so check each against the Iceberg version you run.
  • Upgrade planning: Confirm the release notes for each minor line, since spec-level changes and catalog behavior can differ between versions.

Check the license of every engine, catalog and managed service that reads your tables, since those components carry their own terms even when the format is Apache-2.0.

How much effort does Apache Iceberg vs Apache Hudi take to run?

Can a team run Apache Iceberg or Apache Hudi without outside help?

Yes, many teams run either format on their own, but the answer depends on how much of the stack they already operate. Both are Apache-2.0 projects, so there is no license fee for the format itself. The effort sits in the engines that write and read the tables, the catalog that tracks them, and the maintenance jobs that keep files healthy. A team that already runs Spark or Flink writers, a catalog service and scheduled table maintenance can usually extend that setup. A team building its first lakehouse should weigh whether it has someone who can own compaction, snapshot expiry and catalog access control over time. A fixed-scope engagement can cover the design and first rollout, while the team keeps routine operation.

What skills and time does running Iceberg or Hudi require?

The work needs data engineers who understand table layout, file sizes and write patterns, plus platform staff who run the engines and catalog. Hudi's Merge On Read tables add a background process: the Hudi table types documentation describes periodic compaction that folds log files into new base files, and read-optimized queries only see data up to the last compaction. Hudi's table services, including cleaning, clustering and compaction, can run inline with Spark or Flink writers or be operated independently, which lets a team choose where that load lands. Iceberg's maintenance tasks, such as expiring snapshots and compacting small files, are run by the engine or a scheduled job. Ask any provider to name the people, the on-call model and the scheduled jobs it expects you to own after handover.

How do compaction and delete volume change the effort?

Delete and update volume is the main driver of ongoing effort for both formats. In Hudi Merge On Read tables, updates and deletes land in log files that are merged at query time, so heavy write volume means more compaction work and more frequent tuning of compaction schedules. Hudi Copy On Write tables avoid log files and give fast reads, but writes are slower because updates rewrite base files. Iceberg v2 tables record deletes in delete files, and v3 adds deletion vectors, so the amount of delete activity affects read cost and how often maintenance is needed. Before any engagement, ask for the expected rate of upserts and deletes, the latency your readers need, and a test that measures read and write times at that volume on your own engines.

Can a team move from Hudi to Iceberg later?

A move is possible but it is a migration project, not a configuration change, and its size depends on table count, delete history and downstream consumers. Both formats store data as Parquet files with metadata on top, so the data files themselves are not the main barrier. The cost lies in rebuilding table metadata in the new format, re-pointing writers and readers, retiring Hudi-specific indexes and jobs, and validating results against the old tables. Ask whether the target engines can read both formats during the transition, and whether a parallel run is part of the scope. A written scope with a cutover plan and rollback steps keeps the cost visible before work starts.

What should be confirmed about engine support before choosing?

Confirm support for each engine at the versions you run, not just the format's general claims. The Iceberg project lists Spark, Trino, Flink, Presto, Hive and Impala as engines that can share the same tables, and Trino's Iceberg connector supports Hive metastore, AWS Glue, JDBC, REST, Nessie and Snowflake catalogs. For Hudi, the README lists built-in ingestion for Spark and Flink and a Kafka Connect sink. Check the write path, delete and upsert support, time travel syntax, and catalog compatibility for every engine that will touch the tables. Your engine versions and catalog choice should be tested together before the design is signed off.

What does a fixed-scope implementation cover for this decision?

A fixed-scope engagement with DeployOpen is agreed in writing before work starts, and it covers the deliverables named in that scope. For this decision, a typical scope could include reviewing your write and delete patterns, designing table layouts and maintenance schedules, deploying the chosen engines and catalog in your own infrastructure, setting up backup and access controls, and documenting the runbooks your team will follow. DeployOpen does not issue audit reports or certifications, and it does not run your data as a managed service unless a retainer covers ongoing support, upgrades and continuous testing. Ask for the deliverables, acceptance tests, the engine versions in scope, and what is excluded.

Which costs recur after an Iceberg or Hudi rollout?

The format license is zero cost in both cases, so recurring cost comes from infrastructure and people. The recurring items are compute for compaction and maintenance jobs, storage growth from retained snapshots and log or delete files, catalog hosting, engine upgrades, and the staff time spent on monitoring and tuning. Support retainers, if you buy them, are also recurring and should be priced against the scope they cover. Costs rise with delete volume, number of tables, retention period and the frequency of schema or engine changes. When you request a quote, ask which of these items are one-time, which recur, and which of them you would pay to a cloud provider rather than to DeployOpen.

How DeployOpen helps choose, implement or migrate a table format

DeployOpen approaches Apache Iceberg vs Apache Hudi as a platform engineering decision tied to your own infrastructure, not as a tool preference. We plan, implement, test, document and support the table format inside your cloud accounts, data centers or air-gapped networks, so the choice fits your upsert, delete and change data capture workload.

How we scope

  • Engagements are fixed-scope, with scope and deliverables agreed in writing before work starts.
  • We map which tables need upserts, deletes or change data capture, and which engines must read them.
  • The written scope names the query engines, catalogs and write paths in use, such as Spark, Flink or Trino.
  • Ask us to confirm the Iceberg or Hudi version and table type each engine supports in your environment.

How we build

  • We set up the chosen table format on your storage and catalog, inside your own infrastructure.
  • For Iceberg, we can configure delete handling and compaction jobs against your write volumes and read patterns.
  • For Hudi, we can configure Copy On Write or Merge On Read tables, indexing and compaction schedules as agreed in scope.
  • We test update, delete and incremental read paths against your own data before anything reaches production.

How we hand over

  • Deliverables include documentation of table design, write paths, compaction settings and rollback procedures.
  • We document which engines read each table and the version constraints that apply to each one.
  • Operating runbooks cover routine maintenance, so your platform team can run the tables after handover.
  • Ask for the handover list in the quote so that every deliverable is named and agreed before work begins.

How we support

  • Ongoing support, upgrades and continuous testing are available as retainers after the initial build.
  • Retainers can cover table format upgrades, compaction tuning and engine version changes as your stack evolves.
  • If you later move between Iceberg and Hudi, we scope the migration in writing with its own deliverables.
  • Confirm response expectations and any service levels in the retainer agreement, since none are published here.

Talk to our team.

Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.

  • 30-minute call: free, with no obligation.
  • NDA on request: we can sign before you share details.
  • Clear next steps: a scope and plan after the call.