Help Center/ MapReduce Service/ Component Development Specifications/ Hudi/ Spark on Hudi Development Specifications/ Parameter Specifications for Spark Asynchronous Task Execution Table Compaction
Updated on 2025-04-15 GMT+08:00
View PDF
Share

Parameter Specifications for Spark Asynchronous Task Execution Table Compaction

  • Do not manually execute the run schedule command to generate a compaction plan when the writing job has not stopped.

    Incorrect example:

    run schedule on dsrTable

    If there are other tasks writing to this table, executing this operation will result in data loss.

  • When executing the run compaction command, do not set hoodie.run.compact.only.inline to false. Instead, set it to true.

    Incorrect example:

    set hoodie.run.compact.only.inline=false;
run compaction on dsrTable;

    If there are other tasks writing to this table, executing these operations will result in data loss.

    Correct example: asynchronous compaction

    set hoodie.compact.inline = true;
set hoodie.run.compact.only.inline=true;
run compaction on dsrTable;