Creating a Hudi Table on Flink and Enabling Hidden Partitioning
For table creation statements, see Creating a FlinkServer Job to Interconnect with a Hudi Table.
Add the following hidden partitioning parameters in the Hudi table with clause of Flink SQL:
hoodie.hidden.partitioning.max.rules = [hidden_partition_max_rules_num]
'hoodie.hidden.partitioning.enabled' = [hidden_partition_enable]
'hoodie.hidden.partitioning.rule' = [hidden_partition_rules]
| Parameter | Description | Mandatory |
|---|---|---|
| hoodie.hidden.partitioning.max.rules | Maximum number of hidden partitioning rules allowed. The value is a string. | No. The default value is 5. Performance degrades as the number of partitioning rules increases. If more than five rules are defined, you should adjust this parameter. |
| hidden_partition_rules | Hidden partitioning rule. The value is a string. | Yes |
| hidden_partition_enable | Whether to enable hidden partitioning. The value is of the Boolean type. | Yes |
Examples
create table hudi_source( uuid varchar(20), name varchar(10), age int, ts timestamp(3), ts6 timestamp(6), p varchar(20) ) with ( 'connector' = 'hudi', 'path' = 'hdfs://hacluster/tmp/hudi/flink_stream_mor3', 'table.type' = 'MERGE_ON_READ', 'hoodie.datasource.write.recordkey.field' = 'uuid', 'write.precombine.field' = 'uuid', 'hoodie.datasource.write.hive_style_partitioning' = 'true', 'hoodie.hidden.partitioning.enabled' = 'true', 'hoodie.hidden.partitioning.rule' = 'truncate(partition, 3), date(ts, yyyy), identity(col1)' );
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.