Help Center> Data Lake Factory> User Guide> References> Nodes> MRS Flink Job
Updated on 2022-02-22 GMT+08:00
View PDF

MRS Flink Job

Function

The MRS Flink node is used to execute predefined Flink jobs in MRS.

Parameter

Configure the parameters of the MRS Flink node by referring to Table 1 and Table 2.

Table 1 Property parameters

Parameter

Mandatory

Description

Node Name

Yes

Name of the node. Must consist of 1 to 128 characters and contain only letters, digits, underscores (_), hyphens (-), slashes (/), less-than signs (<), and greater-than signs (>).

MRS Cluster Name

Yes

Select the MRS cluster.

To create an MRS cluster, use either of the following methods:
  • Click . On the Clusters page, create an MRS cluster.
  • Go to the MRS console to create an MRS cluster.

Flink Job Name

Yes

Name of the MRS Flink job. Must consist of 1 to 64 characters and contain only letters, digits, and underscores (_).

Flink Job Resource Package

Yes

Select a JAR package. Before selecting a JAR package, you need to upload the JAR package to the OBS bucket, create a resource on the Resource Management page, and add the JAR package to the resource management list. For details, see Creating a Resource.

Flink Job Parameter

No

Key parameter of the program that executes the Flink job. This parameter is specified by a function in the user program. Separate multiple parameters with space.

Program Parameter

No

Used to configure optimization parameters such as threads, memory, and vCPUs for the job to optimize resource usage and improve job execution performance.

NOTE:

This parameter is mandatory if the cluster version is MRS 1.8.7 or later than MRS 2.0.1.

For details about the program parameters of MRS Flink jobs, see Running a Flink Job in the MapReduce User Guide.

Input Data Path

No

Path where the input data resides.

Output Data Path

No

Path where the output data resides.
Table 2 Advanced settings

Parameter

Mandatory

Description

Max. Node Execution Duration

Yes

Maximum duration of executing a node. When Retry upon Failure is set to Yes for a node, the node can be re-executed for numerous times upon an execution failure within the maximum duration.

Retry upon Failure

Yes

Specifies whether to re-execute a node after the node fails to be executed.

  • Yes: The node will be re-executed after it fails to be executed. The following parameters must be configured:
    • Maximum Retries
    • Retry Interval (seconds)
  • No: The node will not be re-executed. This is the default setting.
NOTE:

If Timeout Interval is configured for the node, the node will not be executed again after the execution times out. Instead, the node is set to the failure state.

Failure Policy

Yes

Policies to be performed after the node fails to be executed:

  • End the current job execution plan
  • Go to the next node
  • Suspend the current job execution plan
  • End execution plans of the current and subsequent nodes
Parent topic: Nodes