Creating a Kafka_SSL Datasource Authentication

Updated on 2024-09-30 GMT+08:00

View PDF

Scenario

Create a Kafka_SSL datasource authentication on the DLI console to store the Kafka authentication information to DLI. This will allow you to access to Kafka instances without having to configure a username and password in SQL jobs.

NOTE:

When Kerberos authentication is enabled for MRS Kafka but SSL authentication is disabled, create a Kerberos authentication. When creating a table, configure krb_auth_name to associate the datasource authentication.
If Kerberos authentication and SSL authentication are both enabled for MRS Kafka, you need to create Kerberos and Kafka_SSL authentications. When creating a table, configure krb_auth_name and ssl_auth_name to associate the datasource authentications.
Datasource authentication is not required when Kerberos authentication is disabled but SASL authentication is enabled for MRS Kafka (for example, when a username and a password are used for PlainLoginModule authentication).
When Kerberos authentication is disabled but SSL authentication is enabled for MRS Kafka, you need to create a Kafka_SSL authentication. When creating a table, configure ssl_auth_name to associate the datasource authentication.
When Kerberos authentication is disabled but SASL authentication and SSL authentication are enabled for MRS Kafka, you need to create a Kafka_SSL authentication. When creating a table, configure ssl_auth_name to associate the datasource authentication.

Data Sources Supported by Kafka_SSL Datasource Authentication

Table 1 lists the data sources supported by Kafka_SSL datasource authentication.

**Table 1** Data sources supported by Kafka_SSL datasource authentication
Job Type	Table Type	Data Source	Notes and Constraints
Flink OpenSource SQL	Source table and result table	Kafka	SASL_SSL authentication has been enabled for DMS Kafka. SASL authentication has been enabled for MRS Kafka. SSL authentication has been enabled for MRS Kafka.

Procedure

Download the authentication credential.
- DMS Kafka
  1. Log in to the DMS (for Kafka) console and click a Kafka instance to access its details page.
  2. In the connection information, find the SSL certificate and click Download.
    Decompress the downloaded kafka-certs package to obtain the client.jks and phy_ca.crt files.
- MRS Kafka
  1. Log in to MRS Manager.
  2. Choose System > Permission > User.
  3. Click More, select Download Authentication Credential, save the file, and decompress it to obtain the truststore file.
Upload the authentication credential to the OBS bucket.

Create a datasource authentication.

Log in to the DLI management console.
Choose Datasource Connections. On the page displayed, click Datasource Authentication.

Click Create.

Configure Kafka authentication parameters according to Table 2.

**Table 2** Parameters
Parameter	Description
Type	Select Kafka_SSL.
Authentication Certificate	Name of the datasource authentication to be created. The name can contain only digits, letters, and underscores (_), but cannot contain only digits or start with an underscore (_). The name can contain a maximum of 128 characters.
Truststore Path	OBS path to which the SSL truststore file is uploaded. For MRS Kafka, enter the OBS path of the Truststore.jks file. For DMS Kafka, enter the OBS path of the client.jks file.
Truststore Password	Truststore password.
Keystore Path	OBS path to which the SSL keystore file (key and certificate) is uploaded.
Keystore Password	Keystore (key and certificate) password.
Key Password	Password of the private key in the keystore file.

Figure 1 Creating a datasource authentication – Kafka_SSL

Access Kafka with SASL_SSL authentication enabled.

When creating a data source, associate the data source with the created datasource authentication to access the data source.

Table 3 lists the fields used to associate with the datasource authentication during table creation.

**Table 3** Fields that are used to associate with Kafka_SSL datasource authentication during table creation
Parameter	Mandatory	Data Type	Description
ssl_auth_name	No	String	This field is used to associate datasource authentications when source, result, and dimension tables are created. Name of the created Kafka_SSL datasource authentication. This configuration is used when SSL is configured for Kafka. If only SSL is used, configure the following parameter: 'properties.security.protocol '= 'SSL'; If SASL_SSL is used, configure the following parameters: 'properties.security.protocol' = 'SASL_SSL', 'properties.sasl.mechanism' ='GSSAPI or PLAIN' 'properties.sasl.jaas.config' = 'org.apache.kafka.common.security.plain.PlainLoginModule required username=\"xxx\" password=\"xxx\";'

For details about how to create a table, see Data Lake Insight Syntax Reference.

Flink OpenSource SQL Syntax Reference: Creating a Kafka Source Table

Parent topic: Using DLI Datasource Authentication to Manage Access Credentials for Data Sources

Previous topic: Creating a Kerberos Datasource Authentication

Next topic: Creating a Password Datasource Authentication

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot