Help Center/ DataArts Fabric/ Developer Guide/ Querying Data/ SQL on Iceberg/ How to Handle Iceberg Table Concurrency Conflicts/ Concurrent Data Modification Conflicts
Updated on 2025-08-25 GMT+08:00
View PDF
Share

Concurrent Data Modification Conflicts

When writing data concurrently to an Iceberg table, two common types of conflicts can arise: concurrent data modification conflict and optimistic locking concurrent commit conflict. This section describes the concurrent data modification conflict.

Job failures due to data conflicts may occur under the following concurrent scenarios:

  • Concurrent execution of DML operations on the same partition, such as performing Update, Delete, Insert Overwrite, or Insert operations on the same partition.
  • Simultaneous execution of Compaction and DML operations.
Currently, Iceberg only supports batch processing scenarios. The following conflicts might be encountered during an Insert Overwrite operation:
  • Data file conflict: For partitioned tables, if during data conflict detection, new data files produced by other committed operations within the partitions targeted by the Insert Overwrite are identified, a data conflict is detected. For non-partitioned tables, if any new data files resulting from other committed operations are recognized during data conflict detection, a data conflict is flagged.
  • Delete file conflict: In the case of partitioned tables, if during data conflict detection, it is found that data files have been removed or delete files have been added within the partitions affected by the Insert Overwrite by other operations, this signifies a data conflict. Similarly, for non-partitioned tables, detecting the removal of data files or the addition of delete files by other operations during data conflict checks indicates a data conflict has occurred.