Updated on 2022-07-04 GMT+08:00

Flink Job Management

On the Job Management page of Flink jobs, you can submit a Flink job. Currently, the following job types are supported:

  • Flink SQL: uses SQL statements to define jobs and can be submitted to any queue.
  • Flink Jar: customizes a JAR package job based on Flink APIs. It can run on dedicated queues.
  • Flink Edge SQL: analyzes edge device data using SQL and can be deployed on edge nodes.

Flink job management provides the following functions:

In addition, you can click Quick Links to switch to the details on User Guide.

Assigning Agency Permissions

Agencies are required for DLI to execute Flink jobs. You can set the agency when logging in to the management console for the first time or go to Global Configurations > Service Authorization to modify the agencies.

The permissions are as follows:

  • Tenant Administrator (global): Tenant Administrator permissions are required to access data from OBS to execute Flink jobs on DLI, for example, obtaining OBS/DWS data sources, log dump (including bucket authorization), checkpointing enabling, and job import and export.

    Due to cloud service cache differences, operations require about 60 minutes to take effect.

  • DIS Administrator permissions are required to use DIS data as the data source of DLI Flink jobs.

    Due to cloud service cache differences, operations require about 30 minutes to take effect.

  • To use CloudTable data as the data source of DLI Flink jobs, CloudTable Administrator permissions are required.

    Due to cloud service cache differences, operations require about 3 minutes to take effect.

  • To use IEF to execute Flink Edge jobs in DLI, Tenant Administrator permissions are required. This permission is also required for running other services that require the Tenant Administrator permission.

    Due to cloud service cache differences, operations require about 3 minutes to take effect.

Job Management page

On the Overview page, click Flink Jobs to go to the SQL job management page. Alternatively, you can click Job Management > Flink Jobs. The Flink Jobs page displays all Flink jobs. If there are a large number of jobs, they will be displayed on multiple pages. DLI allows you to view jobs in all statuses.

Table 1 Job management parameters

Parameter

Description

ID

ID of a submitted Flink job, which is generated by the system by default.

Name

Name of the submitted Flink job.

Type

Type of the submitted Flink job. Including:

  • Flink SQL: Flink SQL job
  • Flink Jar: Custom Flink job

Status

Job statuses, including:

  • Draft
  • Submitting
  • Submission failed
  • Running (The billing starts. After the job is submitted, a normal result is returned.)
  • Running Exception (The billing stops. If an exception occurs during job running, the job is stopped.)
  • Stopping
  • Stopped
  • Stopping failed
  • Creating the savepoint
  • Stopped due to arrears (The billing ends. The job is stopped because the user account is in arrears.)
  • Restoring (recharged jobs) (The account in arrears is recharged, and the job is being restored).
  • Completed

Description

Description of the submitted Flink job.

Username

Name of the user who submits a job.

Created

Time when a job is created. Jobs can be displayed in ascending or descending order of the job creation time.

Started

Time when a Flink job starts to run.

Duration

Time consumed by job running.

Operation

  • Edit: Edit a created job. For details, see Editing a Job.
  • Start: Start and run a job. For details, see Starting a Job.
  • More
    • FlinkUI: After you click this button, the Flink job execution page is displayed.
    • Stop: Stop a job in the Submitting or Running status.
    • Delete: Delete a job.
      NOTE:

      A deleted job cannot be restored. Therefore, exercise caution when deleting a job.

    • Modify Name and Description: You can modify the name and description of a job.
    • Import to Save Point: Import the data exported from the original CS job.
    • Trigger Savepoint: You can click this button for jobs in the Running status to save the job status.
    • Manage Permissions: You can view the user permissions corresponding to the job and grant permissions to other users.