SET/RESET
Function
This command is used to dynamically add, update, display, or reset Hudi parameters without restarting the driver.
Syntax
- Add or update a parameter value:
SET parameter_name=parameter_value
This command is used to add or update the value of parameter_name.
- Display a parameter value:
This command is used to display the value of parameter_name.
- Display session parameters:
This command is used to display all supported session parameters.
- Display session parameters along with usage details:
This command is used to display all supported session parameters and their usage details.
- Reset parameter values:
This command is used to reset all session parameters.
Parameter Description
Parameter |
Description |
---|---|
parameter_name |
Name of the parameter to be dynamically added, updated, or displayed. |
parameter_value |
New value to be set for parameter_name. |
Precautions
The following table lists the properties to be used in the SET or RESET commands.
Property |
Description |
---|---|
hoodie.insert.shuffle.parallelism |
Degree of parallelism (DOP) of Spark shuffle for writing data in insert mode. |
hoodie.upsert.shuffle.parallelism |
DOP of Spark shuffle for writing data in upsert mode. |
hoodie.delete.shuffle.parallelism |
DOP of Spark shuffle for deleting data in delete mode. |
hoodie.sql.insert.mode |
Insert mode. The value can be strict, non-strict, or upsert. |
hoodie.sql.bulk.insert.enable |
Whether to enable bulk insert. |
spark.sql.hive.convertMetastoreParquet |
Converts the parquet table into a data source table for reading. If the provider of Hudi is Hive and Spark SQL or Spark Beeline is used to read data, set this parameter to false. |
Examples
- Add or Update command:
set hoodie.insert.shuffle.parallelism = 100; set hoodie.upsert.shuffle.parallelism = 100; set hoodie.delete.shuffle.parallelism = 100;
- Reset command:
RESET
System Response
- You can view the success result in driver logs.
- You can view the failure result on the UI.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot