Help Center> DataArts Studio> User Guide> DataArts Factory> Node Reference> Data Quality Monitor
Updated on 2024-04-03 GMT+08:00
View PDF

Data Quality Monitor

Functions

The Data Quality Monitor node is used to monitor the quality of running data.

Parameters

Table 1 and Table 2 describe the parameters of the Data Quality Monitor node.

Table 1 Parameters of Data Quality Monitor nodes

Parameter

Mandatory

Description

Node Name

Yes

Name of a node. The name must contain 1 to 128 characters, including only letters, numbers, underscores (_), hyphens (-), slashes (/), less-than signs (<), and greater-than signs (>).

Job Type

Yes

Data quality job. The following options are available:

  • Quality job
  • Comparison job

Quality Job Name

Yes

Name of a quality job created in DataArts Quality. This parameter is mandatory when Job Type is Quality Job. For details about how to create a quality job, see Creating Quality Jobs.

Ignore Quality Job Alarm

Yes

This parameter is mandatory when Job Type is Quality Job.

  • Yes: If the quality job is in the alarm state, the status of the current node is set to successful and the subsequent nodes continue to be executed.
  • No: If the quality job is in the alarm state, the status of the current node is set to failed.

Comparison Job Name

Yes

Name of a comparison job created in DataArts Quality. This parameter is mandatory when Job Type is Comparison Job. For details about how to create a comparison job, see Creating a Comparison Job.

Ignore Comparison Job Alarm

Yes

This parameter is mandatory when Job Type is Comparison Job.

  • Yes: If the comparison job is in the alarm state, the status of the current node is set to successful and the subsequent nodes continue to be executed.
  • No: If the comparison job is in the alarm state, the status of the current node is set to failed.
Table 2 Advanced parameters

Parameter

Mandatory

Description

Max. Node Execution Duration

Yes

Execution timeout interval for the node. If retry is configured and the execution is not complete within the timeout interval, the node will be executed again.

Retry upon Failure

Yes

Whether to re-execute a node if it fails to be executed. Possible values:

  • Yes: The node will be re-executed, and the following parameters must be configured:
    • Retry upon Timeout
    • Maximum Retries
    • Retry Interval (seconds)
  • No: The node will not be re-executed. This is the default setting.
    NOTE:

    If retry is configured for a job node and the timeout duration is configured, the system allows you to retry a node when the node execution times out.

    If a node is not re-executed when it fails upon timeout, you can go to the Default Configuration page to modify this policy.

    Retry upon Timeout is displayed only when Retry upon Failure is set to Yes.

Policy for Handling Subsequent Nodes If the Current Node Fails

Yes

Operation that will be performed if the node fails to be executed. Possible values:

  • Suspend execution plans of the subsequent nodes: stops running subsequent nodes. The job instance status is Failed.
  • End the current job execution plan: stops running the current job. The job instance status is Failed.
  • Go to the next node: ignores the execution failure of the current node. The job instance status is Failure ignored.
  • Suspend the current job execution plan: If the current job instance is in abnormal state, the subsequent nodes of this node and the subsequent job instances that depend on the current job are in waiting state.

Enable Dry Run

No

If you select this option, the node will not be executed, and a success message will be returned.
Parent topic: Node Reference