Creating a SQL Job

Function

This API is used to use the POST method to submit a stream SQL job in JSON format.

URI

  • URI format

    POST /v1.0/{project_id}/sql_job

  • Parameter description
    Table 1 URI parameter description

    Parameter

    Mandatory

    Description

    project_id

    Yes

    Project ID. For details about how to obtain the project ID, see Obtaining a Project ID.

Request

Use the POST method to submit a stream SQL job in JSON format.

  • Example request
    {
    "name": "myjob",
    "desc": "This is a job used for counting characters.",
      "template_id": 100000,
      "cluster_id": 100000,
      "sql_body": "select * from source_table",
      "run_mode": "shared_cluster",
      "spu_number": 2,
      "parallel_number": 1,
      "checkpoint_enabled": false,
      "checkpoint_mode": "exactly_once",
      "checkpoint_interval": 0,
      "obs_bucket": "my_obs_bucket",
      "log_enabled": false,
      "smn_topic": "cs_job_exception",
      "restart_when_exception": false,   
      "idle_state_retention": 0,
      "job_type": "flink_sql_job",
      "edge_group_ids": "62de1e1c-066e-48a8-a79d-f461a31b2ee1,2eb00f85-99f2-4144-bcb7-d39ff47f9002"
    }
  • Parameter description
    Table 2 Request parameters

    Parameter

    Mandatory

    Type

    Description

    name

    Yes

    String

    Job name.

    desc

    No

    String

    Job description.

    template_id

    No

    Int

    Template ID.

    If both template_id and sql_body are specified, sql_body prevails. If template_id is specified but sql_body is not, fill sql_body with the template_id value.

    cluster_id

    No

    Int

    Resource ID of an exclusive cluster. Ensure that the current user has been authorized to use resources of the exclusive cluster.

    sql_body

    No

    String

    Stream SQL statement, which includes at least the following three parts: source, query, and sink.

    run_mode

    Yes

    String

    Job running mode. The options are as follows:

    • shared_cluster: indicates that the job is running on a shared cluster.
    • exclusive_cluster: indicates that the job is running on an exclusive cluster.
    • edge_node: indicates that the job is running on an edge node.

    spu_number

    No

    Int

    Number of SPUs selected for a job.

    parallel_number

    No

    Int

    Number of parallel jobs set by a user.

    checkpoint_enabled

    No

    Boolean

    Whether to enable the automatic job snapshot function.

    • true: indicates to enable the automatic job snapshot function.
    • false: indicates to disable the automatic job snapshot function.
    • Default value: false

    checkpoint_mode

    No

    Int

    Snapshot mode. The values include:

    • ExactlyOnce
    • AtLeastOnce

    checkpoint_interval

    No

    Int

    Snapshot interval, expressed by seconds.

    obs_bucket

    No

    String

    OBS path where users are authorized to save the snapshot. This parameter is valid only when checkpoint_enabled is set to true.

    log_enabled

    No

    Boolean

    Whether to enable the function of uploading job logs to users' OBS buckets.

    smn_topic

    No

    String

    If a job fails to work, CS pushes alarm information to the SMN topic.

    restart_when_exception

    No

    Boolean

    Whether to enable the function of automatically restarting a job upon job exceptions.

    idle_state_retention

    No

    Int

    How long the state of a key is retained without being updated before it is removed in GroupBy or Window.

    job_type

    No

    String

    Job type. This parameter can be set to flink_sql_job or flink_sql_edge_job.

    • If run_mode is set to edge_node, this parameter must be flink_sql_edge_job.
    • If run_mode is set to shared_cluster or exclusive_cluster, this parameter must be flink_sql_job.

    edge_group_ids

    No

    String

    List of edge computing group IDs. Use commas (,) to separate multiple IDs.

    tags

    No

    String

    Job tag.

    sys_enterprise_project_id

    No

    String

    ID of the enterprise project to which a job belongs.

    udf_jar_url

    No

    String

    OBS address where the SQL job UDF JAR file is stored.

    dirty_data_strategy

    No

    String

    Dirty data policy of a job.

    • 2:obs-wan-wulan3/jobs: Save.
    • 1: Trigger a job exception.
    • 0: Ignore.

Response

  • Example response
    {
    "message_id": "CS.10001",
    "message": "Message content",
      "current_time": 1533686888000,
      "payload": {
        "job_id": 50320,
        "status_name": "running",
        "status_desc": "Status description"
      }
    }
  • Parameter description
    Table 3 Response parameters

    Parameter

    Mandatory

    Type

    Description

    message_id

    No

    String

    Message type ID.

    message

    No

    String

    Message content.

    current_time

    No

    Int

    Current time, expressed by milliseconds.

    payload

    No

    None

    Information about the job status.

    job_id

    No

    Long

    Job ID.

    status_name

    No

    String

    Name of job status.

    status_desc

    No

    String

    Description of current job status, including the cause of abnormal status and suggestions.

Status Code

Table 4 describes the status code.

Table 4 Status code

Status Code

Description

200

The Stream SQL job is submitted successfully.

400

The input parameters are invalid.

Error Code

For details, see Error Codes.