Adding and Executing a Job (Deprecated)

job_name

Yes

String

Explanation

Job name

Constraints

N/A

Value range

A cluster name can contain 1 to 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed.

Identical job names are allowed but not recommended.

Default value

N/A

cluster_id

Yes

String

Explanation

Cluster ID

Constraints

N/A

Value range

N/A

Default value

N/A

jar_path

No

String

Explanation

Path of the .jar file or .sql file to be executed.

Constraints

The jar_path parameter is mandatory for MapReduce or Spark jobs.

Value range

• The path can contain a maximum of 1,023 characters, excluding special characters such as ;|&><'$. The address cannot be empty or full of spaces.
• This path must start with "/" or "s3a://". The OBS path does not support files or programs encrypted by KMS.
• Spark Script must end with .sql while MapReduce and Spark JAR must end with .jar. sql and jar are case-insensitive.

Default value

N/A

input

No

String

Explanation

Address for inputting data

Constraints

N/A

Value range

This path must start with "/" or "s3a://". Set this parameter to a correct OBS path. The OBS path does not support files or programs encrypted by KMS.

The value can contain a maximum of 1,023 characters, excluding special characters such as ;|&>'<$, and can be left blank.

Default value

N/A

output

No

String

Explanation

Address for outputting data

Constraints

N/A

Value range

This path must start with "/" or "s3a://". A correct OBS path is required. If the path does not exist, the system automatically creates it.

The value can contain a maximum of 1,023 characters and can be left blank. Special characters such as ;|&>'<$ are not allowed.

Default value

N/A

job_log

No

String

Explanation

Path for storing job logs that record job status.

Constraints

N/A

Value range

This path must start with "/" or "s3a://".

The value can contain a maximum of 1,023 characters and can be left blank. Special characters such as ;|&>'<$ are not allowed.

Default value

N/A

job_type

Yes

Integer

Explanation

Job type code

Constraints

N/A

Value range

• 1: MapReduce
• 2: Spark
• 3: Hive Script
• 4: HiveSQL (not supported currently)
• 5: DistCp, importing and exporting data
• 6: Spark Script
• 7: Spark SQL, submitting SQL statements (not supported currently)

Default value

N/A

file_action

No

String

Explanation

File operation type.

Constraints

N/A

Value range

• export: Export data from HDFS to OBS.
• import: Import data from OBS to HDFS.

Default value

N/A

arguments

No

String

Explanation

Key parameters for program execution.

Constraints

This parameter is specified by a function in your program. MRS is responsible for passing the parameter only.

Value range

The value can contain a maximum of 150,000 characters. Special characters (;|&>'<$!"\) are not allowed. This parameter can be left blank. When entering a parameter containing sensitive information (for example, login password), add an at sign (@) before the parameter name to encrypt the parameter value. This prevents the sensitive information from being persisted in plaintext. Therefore, when you view job information on the MRS, sensitive information will be displayed as asterisks (*). Example: username=xxx @password=yyy

Default value

N/A

hql

No

String

Explanation

Spark SQL statement.

Constraints

The statement needs Base64 encoding and decoding. ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/ is a standard encoding table. MRS uses this table for Base64 encoding. The value of the hql parameter is generated by adding any letter to the beginning of the encoded character string. The Spark SQL statement is generated by decoding the value in the background. Example:

1. Enter Spark SQL statement show tables;.
2. Encode it using ABCDEFGHILKJMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/, and obtain the encoded string c2hvdyB0YWLsZXM7.
3. Prefix the c2hvdyB0YWLsZXM7 string with a random letter (for example, g), and then you get gc2hvdyB0YWLsZXM7.
4. The backend automatically decodes gc2hvdyB0YWLsZXM7 and you obtain Spark SQL statement show tables;.

Value range

N/A

Default value

N/A

hive_script_path

No

String

Explanation

Path of the SQL program.

Constraints

This parameter is required only for Spark Script and Hive Script jobs.

Value range

• The path can contain a maximum of 1,023 characters, excluding special characters such as ;|&><'$. The address cannot be empty or full of spaces.
• The path must start with / or s3a://. The OBS path does not support files or programs encrypted by KMS.
• The path must end with .sql. sql is case-insensitive.

Default value

N/A

templated

Boolean

Explanation

Whether job execution objects are generated by job templates.

Constraints

N/A

Value range

N/A

Default value

N/A

created_at

Long

Explanation

Creation time, which is a 10-bit timestamp.

Constraints

N/A

Value range

N/A

Default value

N/A

updated_at

Long

Explanation

Update time, which is a 10-bit timestamp.

Constraints

N/A

Value range

N/A

Default value

N/A

id

String

Explanation

The job ID.

Constraints

N/A

Value range

N/A

Default value

N/A

tenant_id

String

Explanation

The project ID. For details about how to obtain the project ID, see Obtaining a Project ID.

Constraints

N/A

Value range

N/A

Default value

N/A

job_id

String

Explanation

Job application ID

Constraints

N/A

Value range

N/A

Default value

N/A

job_name

String

Explanation

Job name

Constraints

N/A

Value range

A cluster name can contain 1 to 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed.

Identical job names are allowed but not recommended.

Default value

N/A

input_id

String

Explanation

Data input ID

Constraints

N/A

Value range

N/A

Default value

N/A

output_id

String

Explanation

Data output ID

Constraints

N/A

Value range

N/A

Default value

N/A

start_time

Long

Explanation

Start time of job execution, which is a 10-bit timestamp.

Constraints

N/A

Value range

N/A

Default value

N/A

end_time

Long

Explanation

End time of job execution, which is a 10-bit timestamp.

Constraints

N/A

Value range

N/A

Default value

N/A

cluster_id

String

Explanation

Cluster ID

Constraints

N/A

Value range

N/A

Default value

N/A

engine_job_id

String

Explanation

Workflow ID of Oozie

Constraints

N/A

Value range

N/A

Default value

N/A

return_code

String

Explanation

Returned code for an execution result

Constraints

N/A

Value range

N/A

Default value

N/A

is_public

Boolean

Explanation

Whether a job is public

Constraints

The current version does not support this function.

Value range

N/A

Default value

N/A

is_protected

Boolean

Explanation

Whether a job is protected

Constraints

The current version does not support this function.

Value range

N/A

Default value

N/A

group_id

String

Explanation

Group ID of a job

Constraints

N/A

Value range

N/A

Default value

N/A

jar_path

String

Explanation

Path of the .jar file or .sql file to be executed.

Constraints

N/A

Value range

• The path can contain a maximum of 1,023 characters, excluding special characters such as ;|&><'$. The address cannot be empty or full of spaces.
• This path must start with / or s3a://. The OBS path does not support files or programs encrypted by KMS.
• Spark Script must end with .sql while MapReduce and Spark JAR must end with .jar. sql and jar are case-insensitive.

Default value

N/A

input

String

Explanation

Address for inputting data

Constraints

Set this parameter to a correct OBS path. The OBS path does not support files or programs encrypted by KMS.

Value range

This path must start with / or s3a://.

The value can contain a maximum of 1,023 characters and can be left blank. Special characters such as ;|&>'<$ are not allowed.

Default value

N/A

output

String

Explanation

Address for outputting data

Constraints

N/A

Value range

This path must start with / or s3a://. A correct OBS path is required. If the path does not exist, the system automatically creates it.

The value can contain a maximum of 1,023 characters and can be left blank. Special characters such as ;|&>'<$ are not allowed.

Default value

N/A

job_log

String

Explanation

Path for storing job logs that record job running status.

Constraints

N/A

Value range

This path must start with / or s3a://.

The value can contain a maximum of 1,023 characters and can be left blank. Special characters such as ;|&>'<$ are not allowed.

Default value

N/A

job_type

Integer

Explanation

Job type code

Constraints

Spark and Hive jobs can be added to only clusters that include Spark and Hive components.

Value range

• 1: MapReduce
• 2: Spark
• 3: Hive Script
• 4: HiveSQL (not supported currently)
• 5: DistCp, importing and exporting data
• 6: Spark Script
• 7: Spark SQL, submitting SQL statements (not supported currently)

Default value

N/A

file_action

String

Explanation

File operation type

Constraints

N/A

Value range

• export: Export data from HDFS to OBS.
• import: Import data from OBS to HDFS.

Default value

N/A

arguments

String

Explanation

Key parameters for program execution.

Constraints

This parameter is specified by a function in your program. MRS is responsible for passing the parameter only.

Value range

The value can contain a maximum of 150,000 characters. Special characters (;|&>'<$!"\) are not allowed. This parameter can be left blank. When entering a parameter containing sensitive information (for example, login password), add an at sign (@) before the parameter name to encrypt the parameter value. This prevents the sensitive information from being persisted in plaintext. Therefore, when you view job information on the MRS, sensitive information will be displayed as asterisks (*). For example, username=admin @password=***

Default value

N/A

hql

String

Explanation

Hive&Spark SQL statement.

Constraints

N/A

Value range

N/A

Default value

N/A

job_state

Integer

Explanation

Job status code

Constraints

N/A

Value range

• -1: Terminated
• 1: Starting
• 2: Running
• 3: Completed
• 4: Abnormal
• 5: Error

Default value

N/A

job_final_status

Integer

Explanation

Final job status

Constraints

N/A

Value range

• 0: unfinished
• 1: terminated due to an execution error
• 2: executed successfully
• 3: canceled

Default value

N/A

hive_script_path

String

Explanation

Path of the SQL program.

Constraints

This parameter is required only for Spark Script and Hive Script jobs.

Value range

• The path can contain a maximum of 1,023 characters, excluding special characters such as ;|&><'$. The address cannot be empty or full of spaces.
• The path must start with / or s3a://. The OBS path does not support files or programs encrypted by KMS.
• The path must end with .sql. sql is case-insensitive.

Default value

N/A

create_by

String

Explanation

User ID for creating jobs

This parameter is not used in the current version, but is retained for compatibility with earlier versions.

Constraints

N/A

Value range

N/A

Default value

N/A

finished_step

Integer

Explanation

Number of completed steps

This parameter is not used in the current version, but is retained for compatibility with earlier versions.

Constraints

N/A

Value range

N/A

Default value

N/A

job_main_id

String

Explanation

Main ID of a job

This parameter is not used in the current version, but is retained for compatibility with earlier versions.

Constraints

N/A

Value range

N/A

Default value

N/A

job_step_id

String

Explanation

Step ID of a job

This parameter is not used in the current version, but is retained for compatibility with earlier versions.

Constraints

N/A

Value range

N/A

Default value

N/A

postpone_at

Long

Explanation

Delay time, which is a 10-bit timestamp.

This parameter is not used in the current version, but is retained for compatibility with earlier versions.

Constraints

N/A

Value range

N/A

Default value

N/A

step_name

String

Explanation

Step name of a job

This parameter is not used in the current version, but is retained for compatibility with earlier versions.

Constraints

N/A

Value range

N/A

Default value

N/A

step_num

Integer

Explanation

Number of steps

This parameter is not used in the current version, but is retained for compatibility with earlier versions.

Constraints

N/A

Value range

N/A

Default value

N/A

task_num

Integer

Explanation

Number of tasks This parameter is not used in the current version, but is retained for compatibility with earlier versions.

Constraints

N/A

Value range

N/A

Default value

N/A

update_by

String

Explanation

User ID for updating jobs

Constraints

N/A

Value range

N/A

Default value

N/A

credentials

String

Explanation

Token, which is not supported in the current version.

Constraints

N/A

Value range

N/A

Default value

N/A

user_id

String

Explanation

User ID for creating jobs

This parameter is not used in the current version, but is retained for compatibility with earlier versions.

Constraints

N/A

Value range

N/A

Default value

N/A

job_configs

Map<String,Object>

Explanation

Key-value pair set for saving job running configurations

Constraints

N/A

Value range

N/A

Default value

N/A

extra

Map<String,Object>

Explanation

Authentication information, which is not supported in the current version.

Constraints

N/A

Value range

N/A

Default value

N/A

data_source_urls

Map<String,Object>

Explanation

Data source URL

Constraints

N/A

Value range

N/A

Default value

N/A

info

Map<String,Object>

Explanation

Key-value pair set, containing job running information returned by Oozie.

Constraints

N/A

Value range

N/A

Default value

N/A

200

The job has been added.