Help Center/ MapReduce Service/ Component Operation Guide (LTS) (Ankara Region)/ Using KMS/ Transparent Encryption of Upper-layer Components/ Configuring Transparent Encryption for HBase
Updated on 2024-11-29 GMT+08:00
View PDF
Share

Configuring Transparent Encryption for HBase

Scenario

HDFS implements transparent encryption. After an encrypted partition is configured, the encryption and decryption process is implemented by the HDFS client when you store data in the encrypted partition, which is transparent to upper-layer applications. When storing HBase service data to HDFS, configure the HBase root directory on HDFS as the encrypted partition. Transparent encryption is supported by default.

Prerequisites

  • The KMS, HDFS, and HBase services have been installed and are running properly.
  • The HDFS service has been interconnected with KMS. For details, see Interconnecting HDFS with KMS.
  • The key used for encryption has been created. For details, see Key Management.
  • The cluster client has been installed in a directory, for example, /opt/client.

Procedure

  1. Stop the HBase service.

    Log in to FusionInsight Manager, and choose Cluster > Name of the desired cluster > Services. Click in the upper right corner of the HBase area, and click Stop Service.

  2. Log in to the cluster client and access the client installation directory.

    For example, run the following commands:

    cd /opt/client

  3. Import the environment variables of the client and run the kinit command as user hdfs to authenticate the user.

    source bigdata_env

    kinit hdfs

  4. Move the files in the HBase root directory (specified by hbase.data.rootdirand set to /hbase by default) to the backup directory.

    For example, if the HBase root directory is /hbase and the backup directory path is /hbase_bak, run the following commands:

    hdfs dfs -mkdir -p /hbase_bak/archive

    hdfs dfs -mkdir -p /hbase_bak/data

    hdfs dfs -mv /hbase/archive/* /hbase_bak/archive

    hdfs dfs -mv /hbase/data/* /hbase_bak/data

    hdfs dfs -rm -r /hbase/data

    hdfs dfs -rm -r /hbase/archive

    hdfs dfs -mv /hbase/* /hbase_bak

    • If HBase FileStream (HFS) is configured and transparent encryption is required, repeat 4 to 7 to set the HFS file storage directory (specified by the hbase.filestream.rootdir parameter of the HBase instance) to a partition with transparent encryption enabled.
    • If HDFS Federation has been configured and transparent encryption is required, the backup directory and HBase root directory must be in the same NameService. When you perform 4 to 10, add hdfs:// NameService name (HDFS file system prefix) to the HBase root directory and backup directory, for example, hdfs://ns1/hbase and hdfs://ns1/hbase_bak.

  5. Run the following command to use the generated key to set the HBase root directory as the encrypted partition:

    hdfs crypto -createZone -keyName key_name -path HBase root directory

    hdfs crypto -listZones // Check the encrypted partition.

    For example, run the following commands:

    hdfs crypto -createZone -keyName key1 -path /hbase

    hdfs crypto -listZones

    If HDFS Federation has been configured, run the following commands:

    hdfs crypto -fs hdfs://NameService name -listZones //Check the encrypted partition.

    For example, run the following commands:

    hdfs crypto -fs hdfs://ns1 -listZones //Check the encrypted partition.

  6. Run the following command as user hbase to perform user authentication:

    kinit hbase

  7. Copy the old HBase data to the encrypted partition.

    hadoop distcp -update -delete -prbugpcaxt -skipcrccheck Backup directory HBase root directory

    For example, run the following command:

    hadoop distcp -update -delete -prbugpcaxt -skipcrccheck /hbase_bak /hbase

  8. Log in to FusionInsight Manager and choose Cluster > Name of the desired cluster > Services > HBase > Configurations. On the HBase configuration page, change the value of hadoop.security.crypto.buffer.size to 65536 and save the change.

    If the parameter cannot be found, choose All Configurations > HBase > Customization and manually add a custom parameter to the hadoop.config.expandor parameter.

  9. Start the HBase service.

    Log in to FusionInsight Manager, and choose Cluster > Name of the desired cluster > Services. Click in the upper right corner of the HBase area, and click Start Service.

  10. Run the following command to delete the back directory:

    hdfs dfs -rm -r Backup directory

    For example, run the following command:

    hdfs dfs -rm -r /hbase_bak

    If HFS is configured and transparent encryption is required, delete the backup directory for storing HFS files.