Data Protection Technologies

Updated on 2025-01-09 GMT+08:00

View PDF

Data Storage Security

To prevent your personal data (such as username, password, and mobile number) from being obtained by unauthorized or unauthenticated entities or individuals, DLI encrypts your data during storage and transmission to prevent data leakage.

Data Destruction Mechanism

When you delete a DLI cluster, your personal data stored in the cluster will also be deleted.

When you delete your mobile number and email address on the console and disable message notifications, the mobile number and email address will also be deleted from the database.

Data Transmission Security

Your personal data is encrypted using TLS 1.2 or TLS 1.3 during transmission. All the calls made to DLI APIs use HTTPS to encrypt the transmitted data.

Communication Encryption During Spark Job Transmission

For Spark jobs, you can enable communication encryption by setting the parameters listed in Table 1.

NOTE:

Make sure that you have uploaded the keys and certificates to the specified OBS path and included them in the other dependencies in the job configuration.

**Table 1** Parameters used to configure communication encryption for Spark job transmission
Parameter	Description	Example Value
spark.network.crypto.enabled	This parameter lets you enable or disable communication encryption for data transmission between nodes. When set to true, Spark encrypts all communications between Executors and Drivers and between Executors. This configuration is critical to ensure data transmission security.	true
spark.network.sasl.serverAlwaysEncrypt	This parameter is used to configure whether the server uses encryption to communicate with clients. When set to true, the server requires all clients to use encrypted connections, which improves communication security.	true
spark.authenticate	This parameter is used to configure whether to authenticate the accesses to the components of a Spark application. Enabling authentication blocks unauthorized accesses. Setting to true means enabling authentication.	true

Communication Encryption During Flink Job Transmission

For Flink jobs, you can enable SSL transmission by setting the parameters listed in Table 2.

NOTE:

Enabling SSL for the data transmission streams between Task Managers significantly affects system performance. Thoroughly evaluate both security and performance before you enable it.
Configure certificate file information in Other Dependencies of the job editing page.
The default path for storing dependency files is /opt/flink/usrlib/userData/ in OBS.
Make sure that you have uploaded the keys and certificates to the specified OBS path and included them in the other dependencies in the job configuration.

**Table 2** Parameters used to configure communication encryption for Flink job transmission
Parameter	Description	Mandatory	Example Value
security.ssl.enabled	Whether to enable SSL.	Yes	true
akka.ssl.enabled	Whether to enable Akka SSL.	No	true
blob.service.ssl.enabled	Whether to enable SSL for the Blob channel.	No	true
taskmanager.data.ssl.enable	Whether to enable SSL for communications between TaskManagers.	No	true
security.ssl.algorithms	Whether to configure SSL encryption algorithms.	No	TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
security.ssl.keystore	Path used to store keystore files. flink.keystore indicates the keystore file generated by generate_keystore.sh*.	Yes	/opt/flink/usrlib/userData/flink.keystore
security.ssl.keystore-password	Password for the keystore file, which can be customized	Yes	-
security.ssl.key-password	Password for the SSL key, which can be customized	Yes	-
security.ssl.truststore	Path used to store truststore files. flink.truststore indicates the truststore file generated by generate_keystore.sh*.	Yes	/opt/flink/usrlib/userData/flink.truststore
security.ssl.truststore-password	Password for the truststore file, which can be customized	Yes	-
security.ssl.rest.enabled	Whether to enable SSL/TLS encryption for REST APIs	Yes	false
security.ssl.verify-hostname	Controls whether to check if the host name (hostname) of the peer matches the information in the certificate when establishing an SSL/TLS connection.	No	false
security.ssl.protocol	Protocol version used when SSL/TLS connections are specified	No	TLS 1.2, TLS 1.3
security.ssl.encrypt.enabled	Whether to enable data encryption for communications within a Flink cluster and between Flink and other components	No	false

Example of enabling communication encryption for Flink job transmission:

security.ssl.enabled: true
security.ssl.encrypt.enabled: false
security.ssl.key-password: ***
security.ssl.keystore-password: Admin12!
security.ssl.keystore:  /opt/flink/usrlib/userData/*.keystore
security.ssl.protocol: TLSv1.2
security.ssl.rest.enabled: false
security.ssl.truststore-password: ***
security.ssl.truststore:  /opt/flink/usrlib/userData/*.truststore
security.ssl.verify-hostname: false

Parent topic: Security

Previous topic: Identity Authentication and Access Control

Next topic: Audit and Logging