RUN_TABLE_SERVICE

Updated on 2024-12-13 GMT+08:00

View PDF

NOTE:

This topic is available for MRS 3.3.1-LTS or later only.

Function

This command is used to perform one-click Compaction/Clean/Archive on Hudi MOR tables. Compaction can be executed only on existing schemas.

Executing a Table Service Command for a Single Table

Command format
call run_table_service(table => 'table', clean_hours_retained => 'clean_hours_retained', archive_hours_retained => 'archive_hours_retained')

Description

**Table 1** Parameters
Parameter	Description	Mandatory
table	Table name	Yes
clean_policy	Execution policy used for cleaning.	No. The time policy is used by default.
clean_trigger_strategy	Cleaning interval.	No. By default, a clean operation is triggered once every clean_hours_retained hours.
clean_hours_retained	Time frame within which the data files will be retained	No. The default value is 24 hours.
clean_commits_retained	Number of data files written by commits that are retained during cleaning.	No. The default value is 10.
archive_policy	Execution policy used for archiving.	No. The time policy is used by default.
archive_hours_retained	How many hours in which the timeline will be archived	No. The default value is 24 hours.
max_commit_to_keep	Maximum number of commits metadata files that can be retained during archiving.	No. The default value is 30.
min_commit_to_keep	Minimum number of commits metadata files that can be retained during archiving.	No. The default value is 20.
compact_inline	Whether to enable compaction.	No. The default value is true.
schedule_compact_inline	Whether to perform only the schedule operation after compaction is enabled. CAUTION: If you want the run_table_service command to generate a compaction plan and not to execute it, set schedule_compact_inline to true and run_compact_inline to false.	No. The default value is false.
run_compact_inline	Whether to perform only the run operation after compaction is enabled. CAUTION: By default, the run_table_service command executes only the existing compaction plan in the Hudi table. If you want the run_table_service command to generate a new compaction plan and execute it, set schedule_compact_inline to false and run_compact_inline to false.	No. The default value is true.
compact_max_delta_commits	Number of commits that triggers compaction.	No. The default value is 5.

Example

// Clean and archive based on the time policy.
call run_table_service(table => 'hudi_table', clean_hours_retained => 1, archive_hours_retained => 1)

// Clean and archive data based on the commits policy.
call run_table_service(table => 'hudi_table', clean_policy => 'KEEP_LATEST_COMMITS', clean_trigger_strategy => 'NUM_COMMITS', clean_commits_retained => 2, archive_policy => 'KEEP_META_FILES_BY_COMMITS', min_commit_to_keep => 3, max_commit_to_keep => 4);

Response
You can view the results on the client.

Concurrently Executing Table Service Commands for Multiple Tables

Command format
call run_table_service(tables => 'table1;table2;table3', cleanNums => 'clean_commits_retained', archiveNums=> 'archive_commits_retained')

Description

**Table 2** Description
Parameter	Description	Mandatory
tables	Names of tables where the table service commands will be executed for. Use semicolons (;) to separate multiple table names.	Yes
cleanNums	Number of commits retained during cleaning.	No
archiveNums	Number of commits retained during archiving.	No. If this item is set, the value must be greater than the value of cleanNums.

Example

// Clean and archive data based on the time policy.
call run_table_service(tables => 'tb1;tb2;tb3', cleanNums  => 10, archiveNums => 12)

Response
You can view the results on the client.

Parent topic: Hudi CALL COMMAND Syntax

Previous topic: SHOW_INVALID_PARQUET

Next topic: SYNC_HIVE

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot