Job Development Process

Updated on 2022-09-23 GMT+08:00

View PDF

The job development function provides the following capabilities:

Provides a graphical designer that allows you to quickly build a data processing workflow by drag-and-drop.
Presets multiple job types, such as data integration, computing and analysis, data monitoring, and resource management, and completes complex data analysis and processing based on dependencies between jobs.
Supports various scheduling modes.
Supports job import and export.
Monitors job status and sends job result notifications.
Provides editing locks for collaborative development.
Supports job version management.

Before developing a job, you can learn about the basic job development process.

Figure 1 Job development process

Create a job: Currently, two job types are available: batch and real-time, which are used for batch data processing and real-time connection data processing, respectively. For details, see Creating a Job.
Develop the job: Develop the created job. You can orchestrate and configure nodes. For details, see Developing a Job.
Schedule the job: Configure job scheduling tasks. For details, see Setting Up Scheduling for a Job.
- If the processing mode of a job is batch processing, configure scheduling types for jobs. Three scheduling types are supported: run once, run periodically, and event-based. For details, see Setting Up Scheduling for a Job Using the Batch Processing Mode.
- If the processing mode of a job is real-time processing, configure scheduling types for nodes. Three scheduling types are supported: run once, run periodically, and event-based. For details, see Setting Up Scheduling for Nodes of a Job Using the Real-Time Processing Mode.
Submit a version and unlock the script: After performing this step, the job can be scheduled and modified by other developers. For details, see Submitting a Version and Unlocking the Script.
(Optional) Manage the job: After the job development is complete, you can manage the job as required. For details, see (Optional) Managing Jobs.