Help Center> Cloud Data Migration> User Guide> Job Management> Source Job Parameters> From HTTP/HTTPS
View PDF

From HTTP/HTTPS

When the source link of a job is the HTTP link, configure the source job parameters based on Table 1. Currently, data can only be exported from the HTTP/HTTPS URLs.

Table 1 Parameter description

Parameter

Description

Example Value

File URL

Use the GET method to obtain data from the HTTP/HTTPS URL.

These connectors are used to read files with an HTTP/HTTPS URL, such as reading public files on the third-party object storage system and web disks.

https://bucket.obs.myhuaweicloud.com/object-key

File Format

CDM supports Binary only, which indicates that files (even not in binary format) will be directly transferred.

Binary

Compression Format
Compression format of the source files. The options are as follows:
  • NONE: Files in all formats can be transferred.
  • GZIP: Only files in gzip format can be transferred.
  • ZIP: Only files in Zip format can be transferred.
  • TAR.GZ: Files in TAR.GZ format are transferred.

None

Compressed File Extension

Extension of the files to be decompressed. This parameter is only displayed when Compression Format is not NONE.

The decompression operation is only performed when the filename extension is used in a batch of files. Otherwise, files are transferred in the original format. If you enter * or leave the parameter blank, all files are decompressed.

*

File Separator

File separator. When multiple files are transferred, CDM uses the file separator to identify files. The default value is |. This parameter is not displayed if Pull List File is set to Yes.

|

Query Parameter
  • If you set this parameter to Yes, the name of the objects uploaded to OBS does not include the query parameter.
  • If you set this parameter to No, the name of the objects uploaded to OBS includes the query parameter.

No

Encryption
If the source data is encrypted, CDM can decrypt the data before exporting it. Select whether to decrypt the source data and select a decryption algorithm. The options are as follows:
  • NONE: Export data without decrypting it.
  • AES-256-GCM: The AES 256-bit encryption algorithm is used to encrypt data. Currently, only the AES-256-GCM (NoPadding) encryption algorithm is supported. This parameter is used for encryption at the migration destination and decryption at the migration source.

For details, see Encryption and Decryption During File Migration.

AES-256-GCM

DEK

This parameter is displayed only when Encryption is set to AES-256-GCM. The key consists of 64 hexadecimal numbers and must be the same as the DEK configured during encryption. If the decryption and encryption keys are inconsistent, the system does not report an exception, but the decrypted data is incorrect.

DD0AE00DFECD78BF051BCFDA25BD4E320DB0A7AC75A1F3FC3D3C56A457DCDC1B

IV

This parameter is displayed only when Encryption is set to AES-256-GCM. The initialization vector consists of 32 hexadecimal numbers and must be the same as the IV configured during encryption. If the initialization vectors are inconsistent, the system does not report an exception, but the decrypted data is incorrect.

5C91687BA886EDCD12ACBC3FF19A3C3F

MD5 File Extension

Check whether the files extracted by CDM are consistent with source files. For details, see MD5 Verification.

.md5
Parent topic: Source Job Parameters