文档首页/ 数据湖探索 DLI/ SDK参考/ Python SDK/ Spark作业相关
更新时间：2024-03-06 GMT+08:00
查看PDF
分享

Spark作业相关

完整样例代码和依赖包说明请参考：Python SDK概述。

提交批处理作业

DLI提供执行批处理作业的接口。您可以使用该接口执行批处理作业。示例代码如下：

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
 
def submit_spark_batch_job(dli_client, batch_queue_name, batch_job_info):
    try:
        batch_job = dli_client.submit_spark_batch_job(batch_queue_name, batch_job_info)
    except DliException as e:
        print(e)
        return

    print(batch_job.job_id)
    while True:
        time.sleep(3)
        job_status = batch_job.get_job_status()
        print('Job status: {0}'.format(job_status))
        if job_status == 'dead' or job_status == 'success':
            break

    logs = batch_job.get_driver_log(500)
    for log_line in logs:
        print(log_line)

取消批处理作业

DLI提供取消批处理作业的接口。您可以使用该接口取消批处理作业。若作业已经执行结束或失败则无法取消。示例代码如下：

1
2
3
4
5
6
7
 
def del_spark_batch(dli_client, batch_id): 
    try: 
        resp = dli_client.del_spark_batch_job(batch_id) 
        print(resp.msg) 
    except DliException as e: 
        print(e) 
        return

删除批处理作业

DLI提供删除批处理作业的接口。您可以使用该接口删除批处理作业。示例代码如下： 
def del_spark_batch(dli_client, batch_id):
    try:
        resp = dli_client.del_spark_batch_job(batch_id)
        print(resp.msg)
    except DliException as e:
        print(e)
        return
父主题： Python SDK