Link to HDFS
Description
By creating an HDFS link, you can extract files from or load files to MRS, FusionInsight HD, or Apache Hadoop. Files in CSV, Parquet, and binary formats are supported.
Sample Link
{ "links": [ { "link-config-values": { "configs": [ { "inputs": [ { "name": "linkConfig.hadoopType", "value": "FusionInsight HD" }, { "name": "linkConfig.host", "value": "10.120.205.143" }, { "name": "linkConfig.casPort", "value": "20009" }, { "name": "linkConfig.port", "value": "28443" }, { "name": "linkConfig.authType", "value": "KERBEROS" }, { "name": "linkConfig.user", "value": "admin" }, { "name": "linkConfig.password", "value": "Add password here" }, { "name": "linkConfig.runMode", "value": "STANDALONE" } ], "name": "linkConfig" } ] }, "name": "hdfslink", "connector-name": "hdfs-connector" } ] }
Link Parameters
Parameter |
Mandatory |
Type |
Description |
---|---|---|---|
linkConfig.hadoopType |
Yes |
Enumeration |
Hadoop type. The options are as follows:
|
linkConfig.uri |
No |
String |
NameNode URI required for the link to Apache Hadoop. The format is ip:port. |
linkConfig.host |
No |
String |
IP address of Manager required for the link to MRS or FusionInsight HD |
linkConfig.port |
No |
String |
Port number of Manager required for the link to FusionInsight HD |
linkConfig.casPort |
No |
String |
Port number of CAS Server that connects to FusionInsight HD required for the link to FusionInsight HD |
linkConfig.user |
No |
String |
Username for logging in to Manager. This parameter is not required if the cluster configuration is used. |
linkConfig.password |
No |
String |
Password for logging in to Manager. This parameter is not required if the cluster configuration is used. |
linkConfig.authType |
No |
Enumeration |
Authentication method. The options are as follows:
|
linkConfig.principal |
No |
String |
Account principal required for Kerveros authentication. You can contact the administrator to obtain the account. Before using the cluster configuration, you must set this parameter in cluster configuration management. |
linkConfig.keytab |
No |
FileContent |
Local absolute path of the keytab file required for Kerveros authentication. You can also contact the administrator to obtain the file. Before using the cluster configuration, you must set this parameter in cluster configuration management. |
linkConfig.runMode |
No |
Enumeration |
Running mode of the HDFS link. The options are as follows:
|
linkConfig.properties |
No |
Map |
Properties. You can add configuration properties of the client. Each property must contain a name and a value. |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.