Updated on 2022-11-18 GMT+08:00

Overview of RESTful APIs

Flink has a monitoring API that can be used to query status and statistics of running jobs, as well as recent completed jobs. This monitoring API is used by Apache Flink Dashboard.

The monitoring API is a RESTful API that accepts HTTP GET requests and responds with JSON data. RESTful API is a set of APIs used to log in to the web server. In Flink, web server is a module of JobManager and shares the same process with JobManager. By default, the listening port of web server is 8081. If you want to change the listening port, modify jobmanager.web.port in the flink-conf.yaml file.

The Netty and the Netty Router library are used to handle REST requests and analysis URLs.

RESTful APIs are executed through HTTP requests.

The format of the HTTP requests is http://<JobManager_IP>:<JobManager_Port><Path>

The JobManager_IP indicates the IP address of JobManager, JobManager_Port indicates the listening port of JobManager, and Path indicates the path. For details, see Table 1. For example, http://10.162.181.57:32261/config.

If you want to modify the configuration file flink-conf.yaml of the Flink Client, add to-be-visited IP addresses (separated with commas) in jobmanager.web.allow-access-address and jobmanager.web.access-control-allow-origin parameters.

Table 1 lists all RESTful API paths supported by Flink.

Table 1 Paths supported by Flink

Path

Description

/config

Some information about the monitoring API and the server setup.

/logout

Some information about the logout.

/overview

Simple summary of the Flink cluster status.

/jobs

IDs of the jobs, grouped by status running, finished, failed, canceled.

/jobmanager/config

the configuration of the jobmanager.

/joboverview

Jobs, grouped by status, each with a small summary of its status.

/joboverview/running

Jobs, grouped by status, each with a small summary of its status. The same as /joboverview, but containing only currently running jobs.

/joboverview/completed

Jobs, grouped by status, each with a small summary of its status. The same as /joboverview, but containing only completed (finished, canceled, or failed) jobs.

/jobs/<jobid>

Summary of one job, listing dataflow plan, status, timestamps of state transitions, aggregate information for each vertex (operator).

/jobs/<jobid>/vertices

Currently the same as /jobs/<jobid>.

/jobs/<jobid>/config

The user-defined execution config used by the job.

/jobs/<jobid>/exceptions

The non-recoverable exceptions that have been observed by the job. The truncated flag defines whether more exceptions occurred, but are not listed, because the response would otherwise get too big.

/jobs/<jobid>/accumulators

The aggregated user accumulators plus job accumulators.

/jobs/<jobid>/checkpoints

checkpoint stats for a job.

/jobs/<jobid>/metrics

a job a list of all available metrics.

/jobs/<jobid>/vertices/<vertexid>

Information about one specific vertex, with a summary for each of its subtasks.

/jobs/<jobid>/vertices/<vertexid>/subtasktimes

This request returns the timestamps for the state transitions of all subtasks of a given vertex. These can be used, for example, to create time-line comparisons between subtasks.

/jobs/<jobid>/vertices/<vertexid>/taskmanagers

TaskManager statistics for one specific vertex. This is an aggregation of subtask statistics returned by /jobs/<jobid>/vertices/<vertexid>.

/jobs/<jobid>/vertices/<vertexid>/accumulators

The aggregated user-defined accumulators, for a specific vertex.

/jobs/<jobid>/vertices/<vertexid>/checkpoints

checkpoint stats for a single job vertex.

/jobs/<jobid>/vertices/<vertexid>/backpressure

back pressure stats for a single job vertex and all its sub tasks.

/jobs/<jobid>/vertices/<vertexid>/metrics

a given task of the values for a set of metrics.

/jobs/<jobid>/vertices/<vertexid>/subtasks/accumulators

Gets all user-defined accumulators for all subtasks of a given vertex. These are the individual accumulators that are returned in aggregated form by the request /jobs/<jobid>/vertices/<vertexid>/accumulators.

/jobs/<jobid>/vertices/<vertexid>/subtasks/<subtasknum>

Summary of the current or latest execution attempt of a specific subtask. See below for a sample.

/jobs/<jobid>/vertices/<vertexid>/subtasks/<subtasknum>/attempts/<attempt>

Summary of a specific execution attempt of a specific subtask. Multiple execution attempts happen in case of failure/recovery.

/jobs/<jobid>/vertices/<vertexid>/subtasks/<subtasknum>/attempts/<attempt>/accumulators

The accumulators collected for one specific subtask during one specific execution attempt (multiple attempts happen in case of failure/recovery).

/jobs/<jobid>/plan

The dataflow plan of a job. The plan is also included in the job summary (/jobs/<jobid>).

/taskmanagers

Some information of the TaskManagers.

/taskmanagers/<taskmanagerid>/metrics

the metrics information of a TaskManager.

/taskmanagers/<taskmanagerid>/log

the log information of a TaskManager.

/taskmanagers/<taskmanagerid>/stdout

the stdout of a TaskManager.

/jobmanager/log

the log of Jobmanager.

/jobmanager/stdout

the stdout of Jobmanager.

/jobmanager/metrics

the metrics of Jobmanager.

/:*

services requests to web frontend's static files, such as HTML, CSS, or JS files.

Table 2 describes variables listed in Table 1.

Table 2 Description of variables

Variable

Description

jobid

ID of jobs

vertexid

Vertexes ID of the flow diagram.

subtasknum

Sum of subtasks.

attempt

Times of attempts

taskmanagerid

ID of TaskManager