Developing a Real-Time Processing Single-Task Flink Jar Job

Flink Job Name

Yes

Enter the Flink job name.

The name is automatically generated in Workspace-Job name format.

The job name can contain 1 to 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed.

MRS Cluster

Yes

Select an MRS cluster.

Program Parameter

No

Set job running parameters.

(Optional) Configure optimization parameters such as threads, memory, and vCPUs for the job to optimize resource usage and improve job execution performance.

Click Select Template and select a parameter template. You can also select multiple templates. For details on how to create data connections, see Configuring a Template.

For details about the program parameters of MRS Spark jobs, see Running a Flink Job in the MapReduce Service User Guide.

Job Execution Parameter

No

Set the parameters for the Flink job.

Variables required for executing the Flink job. These variables are specified by the functions in the Hive script. Multiple parameters are separated by spaces.

MRS Resource Queue

No

Select a created MRS resource queue.

Select a queue you configured in the queue permissions of DataArts Security. If you set multiple resource queues for this node, the resource queue you select here has the highest priority.

Flink job resource package

Yes

Select a JAR package. Before selecting a JAR package, upload the JAR package to the OBS bucket, create a resource on the Manage Resource page, and add the JAR package to the resource management list. For details, see Creating a Resource.

Rerun Policy

No

• Rerun from the previous checkpoint
• Rerun the job

Input Data Path

No

Set the input data path. You can select an HDFS or OBS path.

Output Data Path

No

Set the output data path. You can select an HDFS or OBS path.

Job Status Polling Interval (s)

Yes

Set the interval at which the system checks whether the job is complete. The interval can range from 30s to 60s, or 120s, 180s, 240s, or 300s.

During job execution, the system checks the job status at the configured interval.

Maximum Wait Time

Yes

Set the timeout interval for the job. If the job is not complete within the timeout interval and retry is enabled, the job will be executed again.

Retry upon Failure

No

Whether to re-execute a node if it fails to be executed.

• Yes: The node task will be re-executed, and the following parameters must be configured:
  • Retry upon Timeout
  • Maximum Retries
  • Retry Interval (seconds)
• No: The node will not be re-executed. This is the default setting.