Creating a Hudi Table on Spark and Enabling Hidden Partitioning
For details about the table creation statement, see CREATE TABLE.
To create a Hudi table with hidden partitioning, add the following parameters to options_list:
hoodie.hidden.partitioning.max.rules = [hidden_partition_max_rules_num]
hoodie.hidden.partitioning.rule = [hidden_partition_rules]
hoodie.hidden.partitioning.enabled = [hidden_partition_enable]
| Parameter | Description | Mandatory |
|---|---|---|
| hoodie.hidden.partitioning.max.rules | Maximum number of hidden partitioning rules allowed. The value is a string. | No. The default value is 5. Performance degrades as the number of partitioning rules increases. If more than five rules are defined, you should adjust this parameter. |
| hidden_partition_rules | Hidden partitioning rule. The value is a string. | Yes |
| hidden_partition_enable | Whether to enable hidden partitioning. The value is of the Boolean type. | Yes |
Examples
create table test ( id int, name string, price double, size int, ts timestamp ) using hudi tblproperties (primaryKey = 'id', preCombineField = 'ts', hoodie.hidden.partitioning.rule = 'date(ts, yyyy), date(ts, MM), bucket(id, 5)', hoodie.hidden.partitioning.enabled = 'true' );
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.