Updated on 2025-05-07 GMT+08:00

Task Survey

The following describes the task survey.

Table 1 Task survey

Survey Content

Description

Task scheduling

For example, Azkaban, DolphinScheduler, Hera, and Crontab.

Task type

The task is classified based on programming languages:

  • Jar tasks: used in MRS, Flink, and Spark.
  • SQL tasks: used in Hive, Spark, and UDF.
  • Python tasks: used in Spark and algorithm scenarios.
  • Others: used for script calling, such as Shell and Scala.

Task quantity

Survey the total number of all types of tasks to evaluate the task migration period and reconstruction workload. For example, there are 820 Jar tasks on the Azkaban task scheduling platform.

Task update period

Identify the task update periods of different scheduling platforms and task types. For example, the Jar tasks on the Azkaban scheduling platform update monthly, and Shell tasks on the XXL-Job platform update at 22:00 every day.

Task information

Obtain the details of all task for future task reconstruction and migration, including the task ID, name, responsible department, owner, execution time, and update period. Communicate with key personnel in a timely manner.

Task dependency

Identify key tasks and dependencies between tasks.

The survey is performed through the current big data platform with surveys and interviews for supplement and confirmation.