Configuring a Hudi Data Source
Scenario
HetuEngine can be connected to the Hudi data source of the cluster of MRS 3.1.1 or later.
HetuEngine does not support the reading of Hudi bootstrap tables.
Prerequisites
- You have created the proxy user of the Hudi data source. The proxy user is a human-machine user and must belong to the hive group.
- In the /etc/hosts file of all nodes in the cluster where HetuEngine is located, add the mapping between the host names and IP addresses of the cluster where the data source to be connected is located, and add 10.10.10.10 hadoop.System domain name in the /etc/hosts file (for example, 10.10.10.10 hadoop.hadoop.com). Otherwise, HetuEngine cannot connect to the nodes that are not in the cluster based on the host name.
- You have created a HetuEngine administrator by referring to Creating a HetuEngine User.
Procedure
- Perform 1 to 6.g to configure a traditional data source by referring to Configuring a Traditional Data Source.
- In the Custom Configuration area, add a custom parameter, as listed in Table 1.
- Click OK.
Data Type Mapping
Currently, Hudi data sources support the following data types: INT, BIGINT, FLOAT, DOUBLE, DECIMAL, STRING, DATE, TIMESTAMP, BOOLEAN, BINARY, MAP, TRUCT, ARRAY.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.