Scheduling Jobs Across Workspaces

Scenario

If you have assigned permissions based on workspaces, users in different workspaces can only perform operations on jobs in their own workspaces. However, jobs in different workspaces depend on each other. This tutorial describes how to schedule jobs across workspaces.

Solution

The DataArts Studio DataArts Factory module supports job running triggered by events. Therefore, MRS Kafka can be used as the job dependency to implement cross-workspace job scheduling.

As shown in the following figure, after Job 1 in workspace 1 is executed, you can use the Kafka Client to send a message to trigger Job 2; Configure event-triggered scheduling for Job 2, that is, trigger job 2 based on the messages sent by the Kafka Client.

Figure 1 Scheduling solution

Configuration Method

1. Log in to the DataArts Studio console, locate the target DataArts Studio instance, and click Access on the instance card.
2. Locate the row that contains a workspace and click DataArts Factory in the Quick Entry column. On the displayed page, create a job named job_test1. Drag a Dummy and a Kafka Client node and drop them on the canvas, and click and hold to connect the nodes, as shown in Figure 2.

   For the Kafka Client node, you need to select a topic and set Text to the EL expression #{job.name},typeA. After the job in workspace A is executed, Kafka Client will be used to send a single character string message.

   Figure 2 Creating a job named job_test1 in workspace A
   

3. In workspace B, create a job named job_test2. Drag a Dummy and a CDM Job node and drop them on the canvas, and click and hold to connect the nodes, as shown in Figure 3.
   Figure 3 Creating a job named job_test2 in workspace B
   
   • Select Event-based for Scheduling Type, KAFKA for Event Type, and the topic configured for the Kafka Client node in job_test1 in workspace A.
   • Set the IF judgment condition to ensure that the CDM job node is executed only when a message triggering job_test1 is received.
     1. Right-click the connection line and select Set Condition. In the Edit EL Expression dialog box, enter the IF judgment condition in the text box. The IF judgment condition is a ternary expression based on the EL expression syntax. The node following the connection line will be executed only if the result of the ternary expression is true. Otherwise, subsequent nodes will be skipped.

        #{StringUtil.equals(StringUtil.split(Job.eventData,',')[1],'typeA')}

        This IF judgment condition indicates that subsequent job nodes are executed only if typeA follows the comma in the message obtained from Kafka.

     2. Select either Skip all subsequent nodes or Skip the next node for Failure Policy. After the configuration is complete, click OK to save the job.

     

4. Click Test to test the job and view the execution result on the Monitor Instance page. After job_test1 in workspace A is tested, job_test2 in workspace B is triggered.