Setting the Priority for a SQL Job

Scenario

In actual job running, it is necessary to prioritize and ensure the normal running of important and urgent tasks due to their varying levels of importance and urgency. This requires providing the necessary compute resources for their normal operations.

DLI offers a feature to set job priorities for each SQL job, which prioritizes the allocation of compute resources to higher priority jobs when resources are limited.

Notes

• You can assign a priority level of 1 to 10 for each job, with a larger value indicating a higher priority. Compute resources are preferentially allocated to high-priority jobs. That is, if compute resources required for high-priority jobs are insufficient, compute resources for low-priority jobs are reduced.
• Jobs running on a SQL queue have a default priority level of 3.
• To change the priority for a job, you must first stop the job, change the priority level, and then submit the job for the modification to take effect.

Setting the Priority for a SQL Job

spark.sql.dli.job.priority=x

1. Log in to the DLI management console.
2. In the navigation pane on the left, choose Job Management > SQL Jobs.
3. Locate the row containing the job for which you want to set the priority and click Edit in the Operation column.
4. Click Settings. In the Parameter Settings area, configure the spark.sql.dli.job.priority parameter.
   Figure 1 Example configuration for a SQL job