Generating a Static Stream Graph for a Flink SQL Job

Function

This API is used to generate a static stream graph for a Flink SQL job.

Flink 1.15 does not support the generation of static stream graphs.

URI

• URI format

  POST /v3/{project_id}/streaming/jobs/{job_id}/gen-graph

• Parameter description

  Table 1 URI parameter

  Parameter	Mandatory	Type	Description
  project_id	Yes	String	Project ID, which is used for resource isolation. For details about how to obtain its value, see Obtaining a Project ID.

Request

Response

Example Request

Generate a static stream graph for a Flink SQL job.

{
  "job_type": "flink_opensource_sql_job",
  "graph_type": "job_graph",
  "sql_body": "create table orders(\r\n  name string,\r\n  num int\r\n) with (\r\n  'connector' = 'datagen',\r\n  'rows-per-second' = '1', --Generates a data record per second.\r\n  'fields.name.kind' = 'random', --Specify a random generator for the user_id field.\r\n  'fields.name.length' = '5' --Limit the length of user_id to 3.\r\n);\r\n \r\nCREATE TABLE sink_table (\r\n  name string,\r\n  num int\r\n) WITH (\r\n   'connector' = 'print'\r\n);\r\nINSERT into sink_table SELECT * FROM orders;",
  "cu_number": 2,
  "manager_cu_number": 1,
  "parallel_number": 2,
  "tm_cus": 1,
  "tm_slot_num": 0,
  "operator_config": "",
  "static_estimator": true,
  "flink_version": "1.12",
  "static_estimator_config": "{\"operator_list\":[{\"id\":\"0a448493b4782967b150582570326227\",\"output_rate\":1000},{\"id\":\"bc764cd8ddf7a0cff126f51c16239658\",\"output_rate\":1000}]}"
}

Example Response

{
    "message": "",
    "is_success": true,
    "error_code": "",
    "stream_graph": "{\n  \"jid\" : \"44334c4259f6714bddef1ac525364052\",\n  \"name\" : \"InternalJob_1715392878428\",\n  \"nodes\" : [ {\n    \"id\" : \"0a448493b4782967b150582570326227\",\n    \"parallelism\" : 1,\n    \"operator\" : \"\",\n    \"operator_strategy\" : \"\",\n    \"description\" : \"Sink: Sink(table=[default_catalog.default_database.sink_table], fields=[name, num])\",\n    \"chain_operators_id\" : [ \"0a448493b4782967b150582570326227\" ],\n    \"inputs\" : [ {\n      \"num\" : 0,\n      \"id\" : \"bc764cd8ddf7a0cff126f51c16239658\",\n      \"ship_strategy\" : \"FORWARD\",\n      \"exchange\" : \"pipelined_bounded\"\n    } ],\n    \"optimizer_properties\" : {}\n  }, {\n    \"id\" : \"bc764cd8ddf7a0cff126f51c16239658\",\n    \"parallelism\" : 2,\n    \"operator\" : \"\",\n    \"operator_strategy\" : \"\",\n    \"description\" : \"Source: TableSourceScan(table=[[default_catalog, default_database, orders]], fields=[name, num])\",\n    \"chain_operators_id\" : [ \"bc764cd8ddf7a0cff126f51c16239658\" ],\n    \"optimizer_properties\" : {}\n  } ],\n  \"operator_list\" : [ {\n    \"id\" : \"0a448493b4782967b150582570326227\",\n    \"name\" : \"Sink: Sink(table=[default_catalog.default_database.sink_table], fields=[name, num])\",\n    \"type\" : \"Sink\",\n    \"contents\" : \"Sink(table=[default_catalog.default_database.sink_table], fields=[name, num])\",\n    \"parallelism\" : 1,\n    \"tags\" : \"[SINK]\",\n    \"input_operators_id\" : [ \"bc764cd8ddf7a0cff126f51c16239658\" ]\n  }, {\n    \"id\" : \"bc764cd8ddf7a0cff126f51c16239658\",\n    \"name\" : \"Source: TableSourceScan(table=[[default_catalog, default_database, orders]], fields=[name, num])\",\n    \"type\" : \"Source\",\n    \"contents\" : \"TableSourceScan(table=[[default_catalog, default_database, orders]], fields=[name, num])\",\n    \"parallelism\" : 2,\n    \"tags\" : \"[PROCESS, UDF]\",\n    \"input_operators_id\" : [ ]\n  } ]\n}"
}

To make it easier to view the response information, we format stream_graph as follows:

    "jid": "65b6a7b0c1ad95b1722a92b49d2f6eba",
    "name": "InternalJob_1715392245413",
    "nodes": [
        {
            "id": "0a448493b4782967b150582570326227",
            "parallelism": 1,
            "operator": "",
            "operator_strategy": "",
            "description": "Sink: Sink(table=[default_catalog.default_database.sink_table], fields=[name, num])",
            "chain_operators_id": [
                "0a448493b4782967b150582570326227"
            ],
            "inputs": [
                {
                    "num": 0,
                    "id": "bc764cd8ddf7a0cff126f51c16239658",
                    "ship_strategy": "FORWARD",
                    "exchange": "pipelined_bounded"
                }
            ],
            "optimizer_properties": {

            }
        },
        {
            "id": "bc764cd8ddf7a0cff126f51c16239658",
            "parallelism": 2,
            "operator": "",
            "operator_strategy": "",
            "description": "Source: TableSourceScan(table=[[default_catalog, default_database, orders]], fields=[name, num])",
            "chain_operators_id": [
                "bc764cd8ddf7a0cff126f51c16239658"
            ],
            "optimizer_properties": {

            }
        }
    ],
    "operator_list": [
        {
            "id": "0a448493b4782967b150582570326227",
            "name": "Sink: Sink(table=[default_catalog.default_database.sink_table], fields=[name, num])",
            "type": "Sink",
            "contents": "Sink(table=[default_catalog.default_database.sink_table], fields=[name, num])",
            "parallelism": 1,
            "tags": "[SINK]",
            "input_operators_id": [
                "bc764cd8ddf7a0cff126f51c16239658"
            ]
        },
        {
            "id": "bc764cd8ddf7a0cff126f51c16239658",
            "name": "Source: TableSourceScan(table=[[default_catalog, default_database, orders]], fields=[name, num])",
            "type": "Source",
            "contents": "TableSourceScan(table=[[default_catalog, default_database, orders]], fields=[name, num])",
            "parallelism": 2,
            "tags": "[PROCESS, UDF]",
            "input_operators_id": [

            ]
        }
    ]
}

Status Codes

Table 4 describes status codes.

Error Codes

If an error occurs when this API is invoked, the system does not return the result similar to the preceding example, but returns the error code and error information. For details, see Error Codes.