Key Management

Scenario

After the KMS service is installed and connected to a third-party KMS, the HDFS service can use the Transparent Data Encryption (TDE) function.

This section describes how to create a key and encrypt HDFS partitions.

Prerequisites

• The KMS service has been installed and is running properly.
• The user, for example, user1, with the key management permission has been created.
• The cluster client has been installed, for example, in the /opt/kmsclient directory.
• To create an encrypted HDFS partition, you need to install the HDFS client. For example, the installation directory is /opt/hdfsclient, and the hdfstestuser user with the HDFS operation and key management permissions has been created.

Procedure

1. Log in to the cluster client and access the client installation directory.

   cd /opt/kmsclient

2. Import the environment variables of the client and run the kinit command to authenticate the user.

   source bigdata_env

   kinit user1

3. Run the following command to create a key:

   hadoop key create <key_name> [-cipher <cipher>] [-size <size>]

   • key_name: indicates the name of a key.
   • cipher: indicates encryption algorithm (optional). The default value is AES/CTR/NoPadding. You can set it to SM4/CTR/NoPadding.
   • size: indicates the key length (optional). The default value is 128. The AES/CTR/NoPadding encryption algorithms support 128-bit and 256-bit keys. The SM4/CTR/NoPadding encryption algorithms support only 128-bit keys.

   For example, run the following command:

   hadoop key create key1 -cipher SM4/CTR/NoPadding

4. Create an HDFS encryption partition using the HDFS client after the key is created. Data stored in the partition is encrypted for storage.

   cd /opt/hdfsclient

   source bigdata_env

   kinit hdfstestuser

   hdfs dfs -mkdir partition directory // Create a new partition directory.

   hdfs crypto -createZone -keyName key_name -path partition directory //Encrypt the partition.

   hdfs crypto -listZones // Check the encrypted partition.

   Example:

   hdfs dfs -mkdir /secure

   hdfs crypto -createZone -keyName key1 -path /secure

Other Related Commands

You can run the hadoop key -help command to obtain more command parameter descriptions.