Data Protection Technologies

Data Integrity

Data is verified to ensure its integrity during storage and transmission.

User data of MRS is stored in HDFS, which uses CRC32C to verify data. HDFS also supports CRC32 verification, which is much faster than CRC32C. HDFS DataNodes store the verified data. If detecting that the data transmitted from the client is incomplete, they report the exception to the client and notify the client of retransmitting data. The client checks data integrity when reading data from a DataNode. If the data is incomplete, the client will read data from another DataNode.

Data Confidentiality

Based on Apache Hadoop, the distributed file system of FusionInsight MRS provides encrypted storage of files to prevent sensitive data from being stored in plaintext, improving data security.

Applications need to encrypt only specified sensitive data. Services are not affected during the encryption and decryption. In addition to data encryption of the file system, Hive provides column encryption (see Using the Hive Column Encryption Function). Sensitive data can be encrypted and stored after you specify an encryption algorithm during table creation. HBase supports encryption of HFiles and WALs (see HFile and WAL Encryption). You can configure the AES and SMS4 algorithms to encrypt them.

Data Transmission Security

In an MRS cluster, HTTPS encryption is supported for access over web channels. RPC communication supports SASL authentication and supports data encryption using symmetric keys. The encrypted transmission configuration of each component is as follows:

• HDFS encrypted transmission configuration: See Configuring HDFS Data Encryption During Transmission.
• Kafka encrypted transmission configuration: See Configuring Kafka Data Encryption During Transmission.
• Flume encrypted transmission configuration: See Configuring the Encrypted Transmission.
• Flink encrypted transmission configuration: See Encrypted Transmission in Authentication and Encryption.

Data Backup and Disaster Recovery

• Disaster recovery (DR): MRS supports data backup to Huawei Cloud OBS and offers cross-region high reliability.
• Backup: FusionInsight MRS can back up the metadata of OMS, Kafka, DBService, and NameNodes as well as the service data of HDFS, HBase, and Hive.

For details, see Introduction to Backup and Recovery Management.