Help Center/ DataArts Studio/ More Documents/ User Guide (Kuala Lumpur Region)/ User Guide/ DataArts Migration/ Advanced Operations/ Encryption and Decryption During File Migration
Updated on 2023-06-14 GMT+08:00
View PDF
Share

Encryption and Decryption During File Migration

When you migrate files to a file system, CDM can encrypt and decrypt those files. Currently, CDM supports the following encryption modes:

AES-256-GCM

Currently, only AES-256-GCM (NoPadding) is supported. This algorithm is used for encryption at the migration destination and decryption at the migration source. The supported source and destination data sources are as follows:

  • Data sources supported by the migration source: OBS, FTP, SFTP, HDFS (supported in the binary format), and HTTP (applicable to scenarios where OBS shared files are downloaded)
  • Data sources supported by the migration destination: OBS, FTP, SFTP, and HDFS (supported in the binary format)

The following part describes how to use AES-256-GCM to decrypt the encrypted files to be exported from OBS and encrypt the files to be imported to OBS. The methods for using the algorithm on other data sources are the same.

  • Configure decryption at the migration source.
    When you use CDM to create a job for exporting files from OBS, set the migration source to OBS and set the following parameters in the advanced settings of Source Job Configuration:
    1. Encryption: Select AES-256-GCM.
    2. DEK: The key must be the same as that configured in Encryption. Otherwise, the decrypted data is incorrect and the system does not display an error message.
    3. IV: The initialization vector must be the same as that configured in Encryption. Otherwise, the decrypted data is incorrect and the system does not display an error message.

    In this way, after CDM exports encrypted files from OBS, the files written to the migration destination are decrypted plaintext files.

  • Configure encryption at the migration destination.
    When you use CDM to create a job for importing files to OBS, set the migration destination to OBS and set the following parameters in the advanced settings of Destination Job Configuration:
    1. Encryption: Select AES-256-GCM.
    2. DEK: custom encryption key. The key consists of 64 hexadecimal numbers. It is case-insensitive but must contain 64 characters. For example, DD0AE00DFECD78BF051BCFDA25BD4E320DB0A7AC75A1F3FC3D3C56A457DCDC1B.
    3. IV: custom initialization vector. The initialization vector consists of 32 hexadecimal numbers. It is case-insensitive but must contain 32 characters. For example, 5C91687BA886EDCD12ACBC3FF19A3C3F.

    In this way, after CDM imports files to OBS, the files on the migration destination are encrypted using the AES-256-GCM algorithm.

KMS Encryption

The migration source does not support KMS encryption.

CDM supports KMS encryption if tables, files, or a whole database is migrated to OBS. In the Advanced Attributes area of the Destination Job Configuration page, set the parameters.

After KMS encryption is enabled, objects to be uploaded will be encrypted and stored on OBS. When you download the encrypted objects, the encrypted data will be decrypted on the server and displayed in plaintext to users.

  • If KMS encryption is enabled, MD5 verification cannot be used.
  • If the KMS ID of another project is used, change Project ID to the ID of the project to which KMS belongs. If KMS and CDM are in the same project, retain the default value of Project ID.
  • After KMS encryption is performed, the encryption status of the objects on OBS cannot be changed.
  • A key in use cannot be deleted. Otherwise, the object encrypted with this key cannot be downloaded.
Parent topic: Advanced Operations