Operating a Hudi Table Using hudi-cli.sh

Updated on 2024-10-08 GMT+08:00

View PDF

Prerequisites

For a cluster with Kerberos authentication enabled, a user has been created on FusionInsight Manager of the cluster and associated with user groups hadoop and hive.
The Hudi cluster client has been downloaded and installed.

Basic Operations

Log in to the cluster client as user root and run the following commands:
cd {Client installation directory}

source bigdata_env

source Hudi/component_env

kinit Created user
Run the hudi-cli.sh command to access the Hudi client.
cd {Client installation directory}/Hudi/hudi/bin/

./hudi-cli.sh
Run the following example commands as required. For details about all commands, visit the Hudi official website at https://hudi.apache.org/docs/quick-start-guide/.
- Viewing help information
  help // View all Hudi CLI commands.
  
  help 'command' // View the help information and parameter list of a certain command.
- Connecting to a table
  connect --path '/tmp/huditest/test_table'
- Viewing table information
  desc
- Viewing compaction plans
  compactions show all
- Viewing cleaning plans
  cleans show
- Performing the cleaning operation
  cleans run
- Viewing commit information
  commits show
- Viewing the partition where the commit is written to
  commit showpartitions --commit 20210127153356
  
  NOTE:
  
  20210127153356 indicates the commit timestamp.
- Viewing the file where the commit is written to
  commit showfiles --commit 20210127153356
- Comparing the commit information of two tables
  commits compare --path /tmp/hudimor/mytest100
- Rolling back a commit (Only the last commit can be rolled back.)
  commit rollback --commit 20210127164905
- Scheduling a compaction
  compaction schedule --hoodieConfigs 'hoodie.compaction.strategy=org.apache.hudi.table.action.compact.strategy.BoundedIOCompactionStrategy,hoodie.compaction.target.io=1,hoodie.compact.inline.max.delta.commits=1'
- Performing a compaction
  compaction run --parallelism 100 --sparkMemory 1g --retry 1 --compactionInstant 20210602101315 --hoodieConfigs 'hoodie.compaction.strategy=org.apache.hudi.table.action.compact.strategy.BoundedIOCompactionStrategy,hoodie.compaction.target.io=1,hoodie.compact.inline.max.delta.commits=1' --propsFilePath hdfs://hacluster/tmp/default/tb_test_mor/.hoodie/hoodie.properties --schemaFilePath /tmp/default/tb_test_mor/.hoodie/compact_tb_base.json
- Creating a savepoint
  savepoint create --commit 20210318155750
- Rolling back a specified savepoint
  savepoint rollback --savepoint 20210318155750
  CAUTION:
  1. If the commit operation causes metadata conflicts, you can run the commit rollback and savepoint rollback commands to roll back data, but the Hive metadata cannot be rolled back. In this case, you can delete the Hive table and manually synchronize data.
  2. The commit rollback command rolls back only the latest commit, and the savepoint rollback command rolls back only the latest savepoint. You cannot specify a commit or savepoint to roll back.

Parent topic: Using Hudi

Previous topic: Creating a Hudi Table Using Spark Shell

Next topic: Hudi Write Operation

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot