Configuring Minute-based Scheduling

Preparations

• Create a proxy data connection named hive_000001 in Management Center. For details, see MRS Hive Connection Parameters.
• Create a database named default. For details, see Creating a Database.
• Create a Hive SQL script named hive_0520. For details, see Developing an SQL Script.

Procedure

1. In the left navigation pane of DataArts Factory, choose Development > Develop Job.
2. Right-click the directory and select Create Job.
3. Select Batch processing for Job Type and enter job_8750 for Job Name.
4. Set other basic job parameters and click OK to access the job development page.
5. Drag the MRS Hive SQL node to the canvas, click the node name, and set node parameters.

   The key parameters are as follows:

   SQL script: Select the hive_0520 script you have created.

   Script Parameter: Set parameters as needed. If the SQL script does not use any parameters, Script Parameter is not displayed.

   Data Connection: Select the hive_000001 connection you created.

   Database: Select the default database you created.

6. Click Scheduling Setup on the right, set Scheduling Type to Run periodically, and set Scheduling Frequency to Minutes.

   Set Start Time to 00:00, Interval to 50 Minutes, and End Time to 23:59.

   Figure 1 Configuring a job that runs at minute intervals
   
   

   If you select Minutes for Scheduling Frequency, the job cannot be scheduled based on the configured interval, that is, the job cannot be executed at a fixed frequency across hours. The following are examples:

   • A scheduling policy is configured at 14:20 on June 19, 2024. According to the policy, the scheduling starts at 00:30 and ends at 23:59, at an interval of 30 minutes. The job is actually scheduled at 14:30:00, 15:30:00, 16:30:00, 17:30:00, 18:30:00, and more on June 19, 2024.
   • A scheduling policy is configured at 14:20 on June 19, 2024. According to the policy, the scheduling starts at 00:00 and ends at 23:59, at an interval of 50 minutes. The job will be scheduled at 14:50:00, 15:00:00, 15:50:00, 16:00:00, 16:50:00, 17:00:00, 17:50:00, and more on June 19, 2024.

7. Configure dependencies for the job as needed and retain the default values for other parameters.
8. Save and submit the job version.
9. Click Execute to start the job.
10. In the navigation pane on the left, choose Monitoring > Job Monitoring. On the Batch Jobs page, view the job execution result.

Reference

• Creating data connections: You can create data connections by configuring data sources. Based on the data connections in Management Center, DataArts Studio connects to the data lake foundation and provides data development, governance, services, and operations. When the data connection of a script or job is released in the development environment, the connection is automatically switched to the data connection of the production environment. For details, see Creating a DataArts Studio Data Connection.
• Creating tables: You can create tables on the DataArts Factory console, or using DDL or a SQL script. You are advised to create tables on the DataArts Factory console with no code. For details, see Creating a Table.
• Configuring job node parameters: A node defines the actions to be performed on data. DataArts Factory provides nodes used for data integration, compute and analysis, database operations, and resource management. You can choose your desired nodes. For details, see Node Parameter Configuration.
• Dependency policies for periodic scheduling: Currently, DataArts Factory supports two types of dependency policies, that is, dependency between jobs whose scheduling periods are traditional periods and dependency between jobs whose scheduling periods are natural periods. For details, see Dependency Policies for Periodic Scheduling.