Parameter Specifications for Spark Asynchronous Task Execution Table Compaction
- Do not manually execute the run schedule command to generate a compaction plan when the writing job has not stopped.
Incorrect example:
run schedule on dsrTable
If there are other tasks writing to this table, executing this operation will result in data loss.
- When executing the run compaction command, do not set hoodie.run.compact.only.inline to false. Instead, set it to true.
Incorrect example:
Set parameters.
hoodie.run.compact.only.inline=false
Run the following SQL statement:
run compaction on dsrTable;
If there are other tasks writing to this table, executing these operations will result in data loss.
Correct example: asynchronous compaction
hoodie.compact.inline = true hoodie.run.compact.only.inline=true
Run the following SQL statement:
run compaction on dsrTable;
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.