Help Center> Data Lake Insight> User Guide> Job Management> Flink Job Management> Flink Job Management
View PDF

Flink Job Management

On the Job Management page of Flink jobs, you can submit a Flink job. Currently, the following job types are supported:

  • Flink SQL: uses SQL statements to define jobs and can be submitted to any queue.
  • Flink Jar: customizes a JAR package job based on Flink APIs. It can run on dedicated queues.
  • Flink OpenSource SQL: Compatible with the SQL syntax of the community Flink 1.10 version and can run only on CCE queues.
  • Flink Edge SQL: analyzes edge device data using SQL and can be deployed on edge nodes.

Flink job management provides the following functions:

In addition, you can click Quick Links to switch to the details on User Guide.

Assigning Agency Permissions

Agencies are required for DLI to execute Flink jobs. You can set the agency when logging in to the management console for the first time or go to Global Configurations > Service Authorization to modify the agencies.

The permissions are as follows:

  • Tenant Administrator (global): Tenant Administrator permissions are required to access data from OBS to execute Flink jobs on DLI, for example, obtaining OBS/DWS data sources, log dump (including bucket authorization), checkpointing enabling, and job import and export.

    Due to cloud service cache differences, permission setting operations require about 60 minutes to take effect.

  • DIS Administrator permissions are required to use DIS data as the data source of DLI Flink jobs.

    Due to cloud service cache differences, permission setting operations require about 30 minutes to take effect.

  • To use CloudTable data as the data source of DLI Flink jobs, CloudTable Administrator permissions are required.

    Due to cloud service cache differences, permission setting operations require about 3 minutes to take effect.

  • Tenant Administrator (Project-level) permission is needed if you use services, such as IEF, that can run only with this permission.

    Due to cloud service cache differences, permission setting operations require about 3 minutes to take effect.

Job Management page

On the Overview page, click Flink Jobs to go to the SQL job management page. Alternatively, you can click Job Management > Flink Jobs. The Flink Jobs page displays all Flink jobs. If there are a large number of jobs, they will be displayed on multiple pages. DLI allows you to view jobs in all statuses.

Table 1 Job management parameters

Parameter

Description

ID

ID of a submitted Flink job, which is generated by the system by default.

Name

Name of the submitted Flink job.

Type

Type of the submitted Flink job. Including:

  • Flink SQL: Flink SQL job
  • Flink Jar: Custom Flink job
  • Flink OpenSource SQL: Flink OpenSource SQL job
  • Flink Edge SQL: Flink streaming SQL edge job

Status

Job statuses, including:

  • Draft
  • Submitting
  • Submission failed
  • Running (The billing starts. After the job is submitted, a normal result is returned.)
  • Running exception (The billing stops. The job stops running due to an exception.)
  • Downloading
  • Idle
  • Stopping
  • Stopped
  • Stopping failed
  • Creating the savepoint
  • Stopped due to arrears (The billing ends. The job is stopped because the user account is in arrears.)
  • Restoring (recharged jobs) (The account in arrears is recharged, and the job is being restored).
  • Completed

Description

Description of the submitted Flink job.

Username

Name of the user who submits a job.

Created

Time when a job is created. Jobs can be displayed in ascending or descending order of the job creation time.

Started

Time when a Flink job starts to run.

Duration

Time consumed by job running.

Enterprise Project

An enterprise project facilitates project-level management and grouping of cloud resources and users. For details about how to set enterprise projects, see Enterprise Management User Guide.

NOTE:

This parameter is displayed only for users who have enabled the Enterprise Management Service.

Operation
  • Edit: Edit a created job. For details, see Editing a Job.
  • Start: Start and run a job. For details, see Starting a Job.
  • More
    NOTE:

    You can only stop, delete, and modify the name and description of a Flink Edge SQL job.

    • FlinkUI: After you click this button, the Flink job execution page is displayed.
    • Stop: Stop a job in the Submitting or Running status.
    • Delete: Delete a job.
      NOTE:

      A deleted job cannot be restored. Therefore, exercise caution when deleting a job.

    • Modify Name and Description: You can modify the name and description of a job. For details, see Modifying Name and Description.
    • Import to Save Point: Import the data exported from the original CS job. For details, see Importing to a Savepoint.
    • Trigger Savepoint: You can click this button for jobs in the Running status to save the job status. For details, see Triggering a Savepoint.
    • Manage Permissions: You can view the user permissions corresponding to the job and grant permissions to other users. For details, see Managing Flink Job Permissions.
Parent topic: Flink Job Management