Help Center/MapReduce Service/Component Development Specifications/Hudi/Spark on Hudi Stream Scenario Operation Guide/Troubleshooting Methods for Hudi Common Problems/Data Consistency
Updated on 2026-02-25 GMT+08:00
View PDF
Share

Data Consistency

  • The preCombine field must be configured for Hudi tables.

    You can specify the field via preCombineField when creating a Spark table.

  • The partition formats of the Hudi tables must be unified.

    If the same data is written in different partitioning styles, two different partitions are generated.

    hoodie.datasource.write.hive_style_partitioning = false;

    hoodie.datasource.write.hive_style_partitioning = true;

  • The KeyGenerators of Hudi tables must be the same.

    You can check whether the KeyGenerators are the same based on the value of _hoodie_record_key in each data record.

    org.apache.hudi.keygen.ComplexKeyGenerator

    org.apache.hudi.keygen.SimpleKeyGenerator

  • If multiple engines are used to read data from and write data to Hudi tables, you are advised to use lowercase letters for Hudi tables.
  • Execute the table service based on the specifications in Selecting a Proper Table Service Execution Mode. Do not change the specifications without permission.