Updated on 2024-10-23 GMT+08:00

Data Migration

Functions

This node is used to execute an offline or real-time processing migration job.

Parameters

Table 1 and Table 2 describe the parameters of the Data Migration node.

Table 1 Properties

Parameter

Mandatory

Description

Node Name

Yes

Name of a node. The name must contain 1 to 128 characters, including only letters, numbers, underscores (_), hyphens (-), slashes (/), less-than signs (<), and greater-than signs (>).

CDM Cluster Name

Yes

Select CDM clusters. To view the cluster list, click on the right of the drop-down list box. You can select a maximum of 16 clusters.

CDM Job Message Body

Yes

Enter the CDM job message body in JSON format.

To obtain the JSON message body, perform the following steps:

  1. Create a single-task data migration job by referring to Creating an Offline Processing Migration Job.
  2. Press F12 and click the Network tab. Check that the request mode of this job is getPipeline.
    Figure 1 Request mode getPipeline
  3. On the Preview tab page, obtain the JSON message body from the value field in jobBody.
    Figure 2 JSON message body
  4. Copy the obtained message body to CDM Job Message Body. You can edit the JSON message body.
  5. Click Save.
Table 2 Advanced parameters

Parameters

Mandatory

Description

Node Status Polling Interval (s)

Yes

How often the system checks whether the node execution is complete. The value ranges from 1 to 60 seconds.

Max. Node Execution Duration

Yes

Execution timeout interval for the node. If retry is configured and the execution is not complete within the timeout interval, the node will be executed again.

Retry upon Failure

Yes

Whether to re-execute a node if it fails to be executed.

  • Yes: The node task will be re-executed, and the following parameters must be configured:
    • Maximum Retries
    • Retry Interval (seconds)
  • No: The node will not be re-executed. This is the default setting.
NOTE:
  • You are advised to configure automatic retry for only file migration jobs or database migration jobs with Import to Staging Table enabled to avoid data inconsistency caused by repeated data writes.
  • If retry is configured for a job node and the timeout duration is configured, the system allows you to retry a node when the node execution times out.

Policy for Handling Subsequent Nodes If the Current Node Fails

Yes

Operation that will be performed if the node fails to be executed. Possible values:

  • Suspend execution plans of the subsequent nodes: stops running subsequent nodes. The job instance status is Failed.
  • End the current job execution plan: Execution of the current job will stop, and the job instance status will become Failed.
  • Go to the next node: ignores the execution failure of the current node. The job instance status is Failure ignored.
  • Suspend the current job execution plan: If the current job instance is in abnormal state, the subsequent nodes of this node and the subsequent job instances that depend on the current job are in waiting state.

Enable Dry Run

No

If you select this option, the node will not be executed, and a success message will be returned.

Task Groups

No

Select a task group. If you select a task group, you can control the maximum number of concurrent nodes in the task group in a fine-grained manner in scenarios where a job contains multiple nodes, a data patching task is ongoing, or a job is rerunning.