RUN_TABLE_SERVICE

Function

This command is used to perform one-click Compaction/Clean/Archive on Hudi MOR tables. Compaction can be executed only on existing schemas.

Executing a Table Service Command for a Single Table

• Command format

  call run_table_service(table => 'table', clean_hours_retained => 'clean_hours_retained', archive_hours_retained => 'archive_hours_retained')

• Description

  Table 1 Parameters

  Parameter	Description	Mandatory
  table	Table name	Yes
  clean_policy	Execution policy used for cleaning.	No. The time policy is used by default.
  clean_trigger_strategy	Cleaning interval.	No. By default, a clean operation is triggered once every clean_hours_retained hours.
  clean_hours_retained	Time frame within which the data files will be retained	No. The default value is 24 hours.
  clean_commits_retained	Number of data files written by commits that are retained during cleaning.	No. The default value is 10.
  archive_policy	Execution policy used for archiving.	No. The time policy is used by default.
  archive_hours_retained	How many hours in which the timeline will be archived	No. The default value is 24 hours.
  max_commit_to_keep	Maximum number of commits metadata files that can be retained during archiving.	No. The default value is 30.
  min_commit_to_keep	Minimum number of commits metadata files that can be retained during archiving.	No. The default value is 20.
  compact_inline	Whether to enable compaction.	No. The default value is true.
  schedule_compact_inline	Whether to perform only the schedule operation after compaction is enabled. CAUTION: If you want the run_table_service command to generate a compaction plan and not to execute it, set schedule_compact_inline to true and run_compact_inline to false.	No. The default value is false.
  run_compact_inline	Whether to perform only the run operation after compaction is enabled. CAUTION: By default, the run_table_service command executes only the existing compaction plan in the Hudi table. If you want the run_table_service command to generate a new compaction plan and execute it, set schedule_compact_inline to false and run_compact_inline to false.	No. The default value is true.
  compact_max_delta_commits	Number of commits that triggers compaction.	No. The default value is 5.

• Example

  // Clean and archive based on the time policy.
  call run_table_service(table => 'hudi_table', clean_hours_retained => 1, archive_hours_retained => 1)
  
  // Clean and archive data based on the commits policy.
  call run_table_service(table => 'hudi_table', clean_policy => 'KEEP_LATEST_COMMITS', clean_trigger_strategy => 'NUM_COMMITS', clean_commits_retained => 2, archive_policy => 'KEEP_META_FILES_BY_COMMITS', min_commit_to_keep => 3, max_commit_to_keep => 4);

• Response

  You can view the results on the client.

Concurrently Executing Table Service Commands for Multiple Tables

• Command format

  call run_table_service(tables => 'table1;table2;table3', cleanNums => 'clean_commits_retained', archiveNums=> 'archive_commits_retained')

• Description

  Table 2 Description

  Parameter	Description	Mandatory
  tables	Names of tables where the table service commands will be executed for. Use semicolons (;) to separate multiple table names.	Yes
  cleanNums	Number of commits retained during cleaning.	No
  archiveNums	Number of commits retained during archiving.	No. If this item is set, the value must be greater than the value of cleanNums.

• Example

  // Clean and archive data based on the time policy.
  call run_table_service(tables => 'tb1;tb2;tb3', cleanNums  => 10, archiveNums => 12)

• Response

  You can view the results on the client.