HFile and WAL Encryption

HFile and WAL Encryption

1. On any HBase node, run the following commands to create a key file as user omm:

   sh ${BIGDATA_HOME}/FusionInsight_HD_8.1.0.1/install/FusionInsight-HBase-2.2.3/hbase/bin/hbase-encrypt.sh <path>/hbase.jks <type> <length> <alias>

   • /<path>/hbase.jks indicates the path for storing the generated JKS file.
   • <type> indicates the encryption type, which can be SMS4 and AES.
   • <length> indicates the key length. SMS4 supports 16-bit and AES supports 128-bit.
   • <alias> indicate the alias of the key file. When you create the key file for the first time, retain the default value omm.

   For example, to generate an SMS4 encryption key, run the following command:

   sh ${BIGDATA_HOME}/FusionInsight_HD_8.1.0.1/install/FusionInsight-HBase-2.2.3/hbase/bin/hbase-encrypt.sh /home/hbase/conf/hbase.jks SMS4 16 omm

   To generate an AES encryption key, run the following command:

   sh ${BIGDATA_HOME}/FusionInsight_HD_8.1.0.1/install/FusionInsight-HBase-2.2.3/hbase/bin/hbase-encrypt.sh /home/hbase/conf/hbase.jks AES 128 omm

   

   • To ensure operations can be successfully performed, the <path>/hbase.jks directory needs to be created in advance, and the cluster operation user must have the rw permission of this directory.
   • After running the command, enter the same <password> for four times. The password is the same as the one encrypted in 3.

2. Distribute the generated key files to the same directory on all nodes in the cluster and assign read and write permission to user omm.
   

   • Administrators need to select a safe procedure to distribute keys based on the enterprise security requirements.
   • If the key files of some nodes are lost, repeat the step to copy the key files from other nodes.

3. On FusionInsight Manager, choose Cluster > Services > HBase > Configurations. Search for and configure the following parameters:
   • hbase.crypto.keyprovider.parameters.encryptedtext: Set this parameter to a ciphertext password in the format of <encrypted_password>.

     <encrypted_password> indicates the encrypted password generated during the key file creation. The parameter value is displayed in ciphertext. Run the following command as user omm to obtain the related encrypted password on the nodes where HBase service is installed:

     sh ${BIGDATA_HOME}/FusionInsight_HD_8.1.0.1/install/FusionInsight-HBase-2.2.3/hbase/bin/hbase-encrypt.sh

     

     Enter the <password>, which is the same password you entered in 1.

   • hbase.crypto.keyprovider.parameters.uri: Set this parameter to the key path and name in the jceks:// <key_Path_Name> format.

     <key_Path_Name> indicates the path of the key file. For example, if the path of the key file is /home/hbase/conf/hbase.jks, set this parameter to jceks:///home/hbase/conf/hbase.jks.

   • hbase.crypto.key.algorithm: If this parameter is set to SMS4 or AES, HFile content is encrypted using SMS4 or AES.
   • hbase.crypto.wal.algorithm: If this parameter is set to SMS4 or AES, WAL content is encrypted in SMS4 or AES mode.
   • hbase.regionserver.wal.encryption: Set this parameter to true.

4. Click Save. Click Dashboard. In the upper right corner of the page, choose More > Restart Service, enter the password of the current user, and click OK to apply the changes.
5. Create an HBase table through CLI or code and configure the encryption mode to enable encryption. <type> indicates the encryption type, and d indicates the column family.
   • When you create an HBase table through CLI, set the encryption mode to SMS4 or AES for the column family.

     create '<table name>', {NAME => 'd', ENCRYPTION => '<type>'}

   • When you create an HBase table using code, set the encryption mode to SMS4 or AES by adding the following information to the code:

     public void testCreateTable() 
     { 
         String tableName = "user"; 
         Configuration conf = getConfiguration();    
         HTableDescriptor htd = new HTableDescriptor(TableName.valueOf(tableName)); 
          
         HColumnDescriptor hcd = new HColumnDescriptor("d"); 
         //Set the encryption mode to SMS4 or AES.
         hcd.setEncryptionType("<type>"); 
         htd.addFamily(hcd); 
      
         HBaseAdmin admin = null; 
         try 
         { 
             admin = new HBaseAdmin(conf); 
              
             if(!admin.tableExists(tableName)) 
             { 
                 admin.createTable(htd); 
             } 
         } 
         catch (IOException e) 
         { 
             e.printStackTrace(); 
         } 
         finally 
         { 
             if(admin != null) 
             { 
                 try 
                 { 
                     admin.close(); 
                 } 
                 catch (IOException e) 
                 { 
                     e.printStackTrace(); 
                 } 
             } 
         } 
     }

6. You can check whether the encryption configuration is successful by referring to Verifying the Encryption Configuration.
7. If you have configured SMS4 or AES encryption by performing 1 to 4, but do not set the related encryption parameter when creating the table in 5, the inserted data is not encrypted.

   In this case, you can perform the following steps to encrypt the inserted data:

   1. Run the flush command for the table to import the data in the memory to the HFile.

      flush'<table_name>'

   2. Run the following commands to modify the table properties:

      disable'<table_name>'

      alter'<table_name>',NAME=>'<column_name>',ENCRYPTION => '<type>'

      enable'<table_name>'

   3. Insert a new data record and flush the table.
      

      A new data record must be inserted so that the HFile will generate a new HFile and the unencrypted data inserted previously will be rewritten and encrypted.

      put'<table_name>','id2','f1:c1','value222222222222222222222222222222222'

      flush'<table_name>'

   4. Perform the following step to rewrite the HFile:
      major_compact '<table_name>'

      

      This step temporarily disables HBase table services for external systems. Exercise caution when performing this step.

   5. You can perform 6 to check whether the encryption configuration is successful.

Verifying the Encryption Configuration

This operation can be performed only when test data can be written to an empty table.

1. Log in to the node where the client is installed as the client installation user. Switch to the client installation directory, for example, /opt/client.

   cd /opt/client

2. Run the following command to set environment variables:

   source bigdata_env

3. Run the following command to authenticate the current user if Kerberos authentication has been enabled for the current security cluster. The current user must have the permission to read and write HBase tables and the HDFS operation permission.

   kinit Component service user

   If Kerberos authentication is disabled for the cluster, set the Hadoop username.

   export HADOOP_USER_NAME=hbase

4. Run the following command to log in to the HBase client:

   hbase shell

   Run the following command to insert a new data record and flush the table to generate an HFile:

   put'<table_name>','id2','d:c1','value222222222222222222222222222222222'

   flush'<table_name>'

   

   • <table_name> indicates the table configured with SMS4 or AES encryption. For details about how to configure SMS4 or AES encryption, go to 5.
   • d indicates the column family configured with SMS4 or AES encryption. For details about how to configure SMS4 or AES encryption, go to 5.

5. Press Ctrl+C to exit the HBase client.
6. Run the following command to view the directory where the HFile file generated in 4 is stored:

   hdfs dfs -ls

   The file directory format is /hbase/data/<namespace_name>/<table_name>/<region_name>/<columnfamily_name>/<HFile_name>.

   

   If <namespace_name> is not specified during HBase table creation, default is used by default.

   Example:

   /hbase/data/default/create_table/dd61b81b1ba1aad6513b9bdcfd8f871c/d/aa6fe387b27443afaba40f5b584c1fa7

7. Run the following command to view the HFile content:

   hbase hfile -f <HFile path> -p

   

   <HFile path> indicates the directory where the HFile file is located.

   The error message "com.huawei.hadoop.hbase.io.crypto.CryptoRuntimeException" will be displayed in the command output. However, the HBase shell can still read the table data, indicating that the encryption configuration is successful.

Modifying a Key File

During the HFile and WAL Encryption operation, the related key file must be generated and its password must be set to ensure system security. After a period of running, you can replace the key file with a new one to encrypt HFile and WAL.

Modifying a key file has a great impact on the system and will cause data loss in case of any misoperation. You are not advised to perform this operation.

1. Run the following command to generate a new key file as user omm:

   sh ${BIGDATA_HOME}/FusionInsight_HD_8.1.0.1/install/FusionInsight-HBase-2.2.3/hbase/bin/hbase-encrypt.sh <path>/hbase.jks <type> <length> <alias-new>

   • <path>/hbase.jks: indicates the path for storing the generated hbase.jks file. The path and file name must be consistent with those of the key file generated in HFile and WAL Encryption.
   • <alias-new>: indicates the alias of the key file. The alias must be different with that of the old key file.
   • <type>: indicates the encryption type, which can be SMS4 or AES.
   • <length> indicates the key length. SMS4 supports 16-bit and AES supports 128-bit.

   For example, to generate an SMS4 encryption key, run the following command:

   sh ${BIGDATA_HOME}/FusionInsight_HD_8.1.0.1/install/FusionInsight-HBase-2.2.3/hbase/bin/hbase-encrypt.sh /home/hbase/conf/hbase.jks SMS4 16 omm_new

   To generate an AES encryption key, run the following command:

   sh ${BIGDATA_HOME}/FusionInsight_HD_8.1.0.1/install/FusionInsight-HBase-2.2.3/hbase/bin/hbase-encrypt.sh /home/hbase/conf/hbase.jks AES 128 omm_new

   

   • To ensure operations can be successfully performed, the <path>/hbase.jks directory needs to be created in advance, and the cluster operation user must have the rw permission of this directory.
   • After running the command, you need to enter the same <password> for three times. This password is the password of the key file. You can use the password of the old file without any security risk.

2. Distribute the generated key files to the same directory on all nodes in the cluster and assign read and write permission to user omm.
   

   Administrators need to select a safe procedure to distribute keys based on the enterprise security requirements.

3. Log in to FusionInsight Manager, choose Cluster > Services > HBase > Configurations, search for hadoop.config.expandor in the search box, and add the following custom parameters:
   • The name of the new custom parameter is hbase.crypto.master.key.name, and the value is omm_new.
   • The name of the new custom parameter is hbase.crypto.master.alternate.key.name, and the value is omm
   Figure 1 Adding a custom parameter
   

4. Click Save. Click Dashboard. In the upper right corner of the page, choose More > Restart Service, enter the password of the current user, and click OK to apply the changes.
5. Log in to the Hbase client by referring to 1~4 in Verifying the Encryption Configuration and run the major compact command to generate an HFile file based on the new encryption algorithm.

   major_compact '<table_name>'

6. On FusionInsight Manager, choose Cluster > Services > HBase and click the hyperlink on the right of HMaster Web UI. In the Region Servers tab, click Compactions to view the major compact progress.

   

7. When all items in Compaction Progress reach 100% and those in Remaining KVs are 0, run the following command as user omm to destroy the old key file:

   sh ${BIGDATA_HOME}/FusionInsight_HD_8.1.0.1/install/FusionInsight-HBase-2.2.3/hbase/bin/hbase-encrypt.sh <path>/hbase.jks <alias-old>

   • <path>/hbase.jks: indicates the path for storing the generated hbase.jks file. The path and file name must be consistent with those of the key file generated in HFile and WAL Encryption.
   • <alias-old>: indicates the alias of the old key file to be deleted.

   For example:

   sh ${BIGDATA_HOME}/FusionInsight_HD_8.1.0.1/install/FusionInsight-HBase-2.2.3/hbase/bin/hbase-encrypt.sh /home/hbase/conf/hbase.jks omm

   

   To ensure operations can be successfully performed, the <path>/hbase.jks directory needs to be created in advance, and the cluster operation user must have the rw permission of this directory.

8. Repeat 2 and distribute the updated key files again.
9. Delete the HBase self-defined configuration item hbase.crypto.master.alternate.key.name added in 3 from FusionInsight Manager.
10. Repeat 4 for the configuration take effect.