Help Center> Data Lake Insight> API Reference> Spark Job-related APIs> Obtaining the List of Batch Processing Jobs
Updated on 2023-12-14 GMT+08:00

Obtaining the List of Batch Processing Jobs

Function

This API is used to obtain the list of batch processing jobs in a queue of a project.

URI

  • URI format

    GET /v2.0/{project_id}/batches

  • Parameter description
    Table 1 URI parameter

    Parameter

    Mandatory

    Type

    Description

    project_id

    Yes

    String

    Project ID, which is used for resource isolation. For details about how to obtain its value, see Obtaining a Project ID.

    Table 2 query parameter description

    Parameter

    Mandatory

    Type

    Description

    job_name

    No

    String

    Name of a batch processing job.

    job-id

    No

    String

    This API is used to query a batch job based on the job ID.

    cluster_name

    No

    String

    DLI queue name. If this parameter is left blank, the names of all batch processing jobs in the current project are obtained. You are advised to specify this parameter, instead of leaving it blank.

    queue_name

    No

    String

    DLI queue name. You can query batch jobs based on the queue name. This method is recommended.

    from

    No

    Integer

    Index number of the start batch processing job. By default, the index number starts from 0.

    size

    No

    Integer

    Number of batch processing jobs to be queried

    state

    No

    String

    Query batch jobs by job status.

    owner

    No

    String

    User who submits a job.

Request

None

Response

Table 3 Response parameters

Parameter

Mandatory

Type

Description

from

No

Integer

Index number of the start batch processing job.

total

No

Integer

Total number of batch processing jobs.

sessions

No

Array of objects

Batch job information. For details, see Table 4.

create_time

No

Long

Time when a batch processing job is created.

Table 4 sessions parameters

Parameter

Mandatory

Type

Description

duration

No

Long

Job running duration (unit: millisecond)

id

No

String

ID of a batch processing job.

state

No

String

Status of a batch processing job

appId

No

String

Back-end application ID of a batch processing job

log

No

Array of Strings

Last 10 records of the current batch processing job

sc_type

No

String

Type of a computing resource. If the computing resource type is customized, value CUSTOMIZED is returned.

cluster_name

No

String

Queue where a batch processing job is located.

create_time

No

Long

Time when a batch processing job is created. The timestamp is in milliseconds.

name

No

String

Name of a batch processing job.

owner

No

String

Owner of a batch processing job.

proxyUser

No

String

Proxy user (resource tenant) to which a batch processing job belongs.

kind

No

String

Type of a batch processing job. Only Spark parameters are supported.

queue

No

String

Queue where a batch processing job is located.

image

No

String

Custom image. The format is Organization name/Image name:Image version.

This parameter is valid only when feature is set to custom. You can use this parameter with the feature parameter to specify a user-defined Spark image for job running.

req_body

No

String

Request parameter details.

update_time

No

Long

Time when a batch processing job is updated. The timestamp is in milliseconds.

Example Request

None

Example Response

{
    "from": 0,
    "total": 1,
    "sessions": [
        {
            "id": "178fa687-2e8a-41ed-a439-b00de60bb176",
            "state": "dead",
            "appId": null,
            "log": [
             "stdout: ",
             "stderr: ",
             "YARN Diagnostics: "
           ],
           "sc_type": "A",
           "cluster_name": "test",
           "create_time": 1531906043036
        }
    ]
}

Status Codes

Table 5 describes the status code.

Table 5 Status codes

Status Code

Description

200

The query is successful.

400

Request error.

500

Internal service error.

Error Codes

If an error occurs when this API is invoked, the system does not return the result similar to the preceding example, but returns the error code and error information. For details, see Error Codes.