Help Center/ MapReduce Service/ Component Operation Guide (LTS)/ Using Hudi/ Hudi Data Management and Maintenance/ Hudi Compaction
Updated on 2026-06-27 GMT+08:00
View PDF
Share

Hudi Compaction

Scenarios

Hudi compaction merges base and log files in MOR tables to optimize data files, improving query performance and storage efficiency.

For MOR tables, data is stored in columnar Parquet files and row-based Avro files, updates are recorded in incremental files, and then a synchronous or asynchronous compaction is performed to generate new versions of columnar files. MOR tables can reduce data ingestion latency, so an asynchronous compaction that does not block ingestion is useful.

Performing Compaction

Compaction can be synchronous or asynchronous.

The synchronization mode is controlled by the hoodie.compact.inline parameter. The default value is true, indicating that the compaction scheduling plan is automatically generated and compaction is executed.
  • Disable synchronous compaction.

    When a data source is written, run the .option("hoodie.compact.inline", "false") command to disable automatic compaction.

    When you use Spark SQL to write data, run the set hoodie.compact.inline=false; command to disable automatic compaction.

  • Only compaction scheduling is generated synchronously, but compaction is not executed.
    • Configure the following option parameters to write to a data source: 
      option("hoodie.compact.inline", "true").
option("hoodie.schedule.compact.only.inline", "true").
option("hoodie.run.compact.only.inline", "false").
    • When you use·Spark SQL to write data, configure the following set parameters: 
      set hoodie.compact.inline=true;
set hoodie.schedule.compact.only.inline=true;
set hoodie.run.compact.only.inline=false;
Perform asynchronous compaction in two steps:
  1. Scheduling a compaction: A compaction is completed by the job of importing data into the data lake. In this step, Hudi scans partitions and selects the file slices to be compacted. A compaction plan is finally written to the Hudi timeline.
  2. Executing a compaction: A separate process or thread reads the compaction plan and performs the compaction of file slices.

Spark SQL implements asynchronous mode. For details, see COMPACTION.

To execute an existing compaction scheduling plan without generating a new one during asynchronous compaction, run the set commands to configure the following parameters:

set hoodie.compact.inline=true;
set hoodie.schedule.compact.only.inline=false;
set hoodie.run.compact.only.inline=true;

For more compaction parameters, see Compaction and Cleaning Configurations.

To ensure the maximum efficiency of data import into the lake, you are advised to generate compaction scheduling plans synchronously and execute compaction scheduling plans asynchronously.