Updated on 2024-08-30 GMT+08:00

To HBase/CloudTable

If the destination link of a job is an HBase link or CloudTable link, configure the destination job parameters based on Table 1.

Table 1 Parameter description

Parameter

Description

Example Value

Table Name

Name of the HBase table to which data will be written. If you want to create an HBase table, you can copy the field names from the migration source. Click the icon next to the text box. The dialog box for selecting the table is displayed.

This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. For details, see Incremental Synchronization Using the Macro Variables of Date and Time.

NOTE:

If you have configured a macro variable of date and time and schedule a CDM job through DataArts Studio DataArts Factory, the system replaces the macro variable of date and time with (Planned start time of the data development jobOffset) rather than (Actual start time of the CDM jobOffset).

TBL_2

Clear Data Before Import

Whether the data in the destination table is cleared before data import. The options are as follows:
  • Yes: The data is cleared.
  • No: The data is not cleared. Instead, it will be added to the existing table.

Yes

Auto Table Creation

This parameter is displayed only when the source is a relational database. The options are as follows:
  • Non-auto creation: CDM will not automatically create a table.
  • Auto creation: If the destination database does not contain the table specified by Table Name, CDM will automatically create the table. If the table specified by Table Name already exists, no table is created and data is written to the existing table.
    NOTE:

    The automatically created HBase table contains the column family and coprocessor information. For other attributes, default values are retained.

Non-auto creation

Rowkey Delimiter

(Optional) Used to combine multiple columns as a rowkey. Spaces are used by default.

,

Rowkey Data Redundancy

(Optional) Whether to write the rowkey data into HBase columns. The default value is No.

No

Compression Format

(Optional) Compression format used in creating an HBase table. The default value is None.
  • None: The files are not compressed.
  • Snappy: The files are compressed in snappy format.
  • gzip: The files are compressed in gzip format.

None

Write WAL

Whether to enable Write Ahead Log (WAL) of HBase. The options are as follows:

  • Yes: If the HBase server breaks down after the function is enabled, you can replay the operations that have not been performed in WAL.
  • No: If you set this parameter to No, the write performance is improved. However, if the HBase server breaks down, data may be lost.

No

Match Data Type

  • Yes: Data of the Short, Int, Long, Float, Double, and Decimal columns in the source database is converted into Byte[] arrays (binary) and written into HBase. Other types of data are written as character strings. If several types of data mentioned above are combined as rowkeys, they will be written as character strings.

    This function saves storage space. In specific scenarios, the rowkey distribution is evener.

  • No: All types of data in the source database are written into HBase as character strings.

No