Overview

Updated on 2025-02-14 GMT+08:00

View PDF

What Is Datasource Authentication?

When analyzing across multiple sources, you are advised not to configure authentication information directly in a job as it can lead to password leakage. Instead, you are advised to use either Data Encryption Workshop (DEW) or datasource authentication provided by DLI to securely store data source authentication information.

DEW is a comprehensive cloud-based encryption service that addresses data security, key security, and complex key management issues. You are advised to use DEW to store authentication information for data sources.
You are advised to use DEW to store authentication information of data sources when Spark 3.3.1 or later and Flink 1.15 or later jobs access data sources using datasource connections. This will help you address issues related to data security, key security, and complex key management. For details, see Using DEW to Manage Access Credentials for Data Sources.
Datasource authentication is used to manage authentication information for accessing specified data sources. After datasource authentication is configured, you do not need to repeatedly configure data source authentication information in jobs, improving data source authentication security while enabling DLI to securely access data sources.
When SQL and Flink 1.12 jobs access data sources using datasource connections, use DLI's datasource authentication feature to manage data source access credentials.

This section describes how to use datasource authentication provided by DLI.

Notes and Constraints

**Table 1** Notes and constraints on datasource authentication
Item	Description
Use case	Only Spark SQL and Flink OpenSource SQL 1.12 jobs support datasource authentication. Flink jobs can use datasource authentication only on queues created after May 1, 2023.
Datasource authentication type	DLI supports four types of datasource authentication. Select an authentication type specific to each data source. CSS: applies to 6.5.4 or later CSS clusters with the security mode enabled. Kerberos: applies to MRS security clusters with Kerberos authentication enabled. Kafka_SSL: applies to Kafka with SSL enabled. Password: applies to GaussDB(DWS), RDS, DDS, and DCS.

Datasource Authentication Types

DLI supports four types of datasource authentication. Select an authentication type specific to each data source.

CSS: applies to 6.5.4 or later CSS clusters with the security mode enabled. During the configuration, you need to specify the username, password, and authentication certificate of the cluster and store the information in DLI through datasource authentication so that DLI can securely access CSS data sources. For details, see Creating a CSS Datasource Authentication.
Kerberos: applies to MRS security clusters with Kerberos authentication enabled. During the configuration, you need to specify MRS cluster authentication credentials, including the krb5.conf and user.keytab files. For details, see Creating a Kerberos Datasource Authentication.
Kafka_SSL: applies to Kafka with SSL enabled. During the configuration, you need to specify the KafkaTruststore path and password. For details, see Creating a Kafka_SSL Datasource Authentication.
Password: applies to GaussDB(DWS), RDS, DDS, and DCS data sources. During the configuration, you need to store the passwords of the data sources in DLI. For details, see Creating a Password Datasource Authentication.

Jobs That Can Connect to Data Sources Through Datasource Authentication

Different types of jobs can connect to data sources through different types of datasource authentication.

For details about the data sources that Spark SQL jobs can connect to through datasource authentication and their constraints, see Table 2.
For details about the data sources that Flink OpenSource SQL 1.12 jobs can connect to through datasource authentication and their constraints, see Table 3.

**Table 2** Data sources that Spark SQL jobs can connect to through datasource authentication
Datasource Authentication Type	Data Source	Notes and Constraints
CSS	CSS	The CSS cluster version must be 6.5.4 or later. The security mode has been enabled for the CSS cluster.
Password	GaussDB(DWS), RDS, DDS, and Redis	-

**Table 3** Data sources that Flink OpenSource SQL 1.12 jobs can connect to through datasource authentication
Table Type	Datasource Authentication Type	Data Source	Notes and Constraints
Source table	Kerberos	HBase	Kerberos authentication has been enabled for the MRS cluster.
	Kerberos	Kafka	Kerberos authentication has been enabled for MRS Kafka.
	Kafka_SSL	Kafka	SASL_SSL authentication has been enabled for DMS Kafka. SASL authentication has been enabled for MRS Kafka. SSL authentication has been enabled for MRS Kafka.
	Password	GaussDB(DWS), RDS, and Redis	-
Result table	Kerberos	HBase	Kerberos authentication has been enabled for the MRS cluster.
	Kerberos	Kafka	Kerberos authentication has been enabled for MRS Kafka.
	Kafka_SSL	Kafka	SASL_SSL authentication has been enabled for DMS Kafka. SASL authentication has been enabled for MRS Kafka. SSL authentication has been enabled for MRS Kafka.
	Password	GaussDB(DWS), RDS, CSS, and Redis	-
Dimension table	Kerberos	HBase	Kerberos authentication has been enabled for the MRS cluster.
Dimension table	Password	GaussDB(DWS), RDS, and Redis	-