Help Center/ Data Lake Insight/ User Guide (Paris Region)/ Job Management/ Flink Job Management/ Job Details
Updated on 2022-07-04 GMT+08:00
View PDF
Share

Job Details

After creating a job, you can view the job details to learn about the following information:

Viewing Job Details

This section describes how to view job details. After you create and run a job, you can view job details, including SQL statements and parameter settings. For a user-defined job, you can only view its parameter settings.

  1. In the left navigation pane of the DLI management console, choose Job Management > Flink Jobs. The Flink Jobs page is displayed.
  2. Click the name of the job to be viewed. The Job Details page is displayed.

    On the Job Details page, you can view SQL statements, configured parameters.

    The following uses a Flink SQL job as an example.
    Table 1 Description

    Parameter

    Description

    Job Status

    The current status of the job is displayed on the right of the job name.

    ID

    Job ID.

    Job Type

    Job type, for example, Flink SQL.

    Queue

    If the queue to which the job belongs is a shared queue, the shared queue is displayed.

    If the queue to which the job belongs is a user-defined dedicated queue, the queue name is displayed.

    Running Mode

    If you create a job in a shared queue, this parameter is Shared.

    If you create a job in a user-defined queue, this parameter is Exclusive.

    CUs

    Number of CUs configured for a job.

    Job Manager CUs

    Number of job manager CUs configured for a job.

    Max Concurrent Jobs

    Maximum number of jobs that can be concurrently executed by a Flink job.

    Save Job Log

    Enabled or disabled.

    Auto Restart upon Exception

    Enabled or disabled.

    Restore Job from Checkpoint

    Enabled or disabled.

    CU(s) per TM

    Number of CUs occupied by each Task Manager configured for a job.

    Slot(s) per TM

    Number of Task Manager slots configured for a job.

    Enable Checkpointing

    Enabled or disabled.

    Checkpoint Interval

    Interval between storing intermediate job running results to OBS.

    Checkpoint Mode

    Checkpoint mode. Values include:

    • At least once: Events are processed at least once.
    • Exactly once: Events are processed only once.

    Idle State Retention Time

    Defines for how long the state of a key is retained without being updated before it is removed in GroupBy or Window.

Checking the Job Monitoring Information

You can use Cloud Eye to view details about job data input and output.

  1. In the left navigation pane of the DLI management console, choose Job Management > Flink Jobs. The Flink Jobs page is displayed.
  2. Click the name of the job to be viewed. The Job Details page is displayed.

    Click Job Monitoring in the upper right corner of the page to switch to the Cloud Eye console.

    The following table describes monitoring metrics related to Flink jobs.

    Table 2 Monitoring metrics related to Flink jobs

    Name

    Description

    Flink job data read rate

    Displays the data input rate of a Flink job for monitoring and debugging. Unit: record/s.

    Flink job data write rate

    Displays the data output rate of a Flink job for monitoring and debugging. Unit: record/s.

    Flink job total data read

    Displays the total number of data inputs of a Flink job for monitoring and debugging. Unit: records

    Flink job total data write

    Displays the total number of output data records of a Flink job for monitoring and debugging. Unit: records

    Flink job byte read rate

    Displays the number of input bytes per second of a Flink job. Unit: byte/s

    Flink job byte write rate

    Displays the number of output bytes per second of a Flink job. Unit: byte/s

    Flink job total read byte

    Displays the total number of input bytes of a Flink job. Unit: byte

    Flink job total write byte

    Displays the total number of output bytes of a Flink job. Unit: byte

    Flink job CPU usage

    Displays the CPU usage of Flink jobs. Unit: %

    Flink job memory usage

    Displays the memory usage of Flink jobs. Unit: %

Viewing Job Running Logs

You can view the run logs to locate the faults occurring during job running.

  1. In the left navigation pane of the DLI management console, choose Job Management > Flink Jobs. The Flink Jobs page is displayed.
  2. Click the name of the job to be viewed. The Job Details page is displayed.
  3. On the Run Log tab page, you can view the Job Manager and Task Manager information of the running job.

    Information about Job Manager and Task Manager is updated every minute. By default, only the run logs generated within the last 1 minute are displayed. You can click Log history to view more logs.

    If you select an OBS bucket for saving job logs during the job configuration, you can switch to the OBS bucket and download log files to view more historical logs.

    If the job is not running, information on the Task Manager page cannot be viewed.

Parent topic: Flink Job Management