Invoking DataArts Quality Operators Using DataArts Factory and Transferring Quality Parameters During Job Running

Scenario

Parameters need to be transferred to a data quality operator job, and the job can run properly.

Procedure

Creating a Quality Job

1. On the DataArts Studio console, locate an instance and click Access. On the displayed page, locate a workspace and click DataArts Quality.
2. (Optional) In the left navigation pane, choose Quality Monitoring > Quality Jobs and create a directory. If a directory exists, you do not need to create one.
3. On the Quality Jobs page, click Create. On the displayed Set Basic Info page, configure basic information about the quality job.
4. Click Next. On the Define Rule page, configure rules for the quality job. In the Object Scope area, set Scanning Scope to Partial and configure parameters.
   Figure 1 Configuring parameters for the data quality job
   

5. Click Next and configure alarm, subscription, and scheduling information. For details about how to configure a quality job, see Creating Quality Jobs.
6. Click Submit. The data quality job is configured successfully.

Configuring a Data Development Job

1. Log in to the DataArts Studio console. Locate an instance and click Access. On the displayed page, locate a workspace and click DataArts Factory.
2. In the left navigation pane of the DataArts Factory console, choose Data Development > Develop Job.
3. Create a batch processing pipeline job and open the job.
4. Drag a Data Quality Monitor operator to the canvas and configure node properties.
   Figure 2 Configuring properties for the Data Quality Monitor node
   

5. Configure the scheduling information.
   Figure 3 Configuring the scheduling information
   

6. Submit the version and perform scheduling.
7. On the Monitor Job page, view the job run logs.
   Figure 4 Viewing run logs