Help Center/ DataArts Studio/ User Guide/ DataArts Factory/ Node Reference/ Shell
Updated on 2024-08-30 GMT+08:00
View PDF
Share

Shell

Functions

The Shell node is used to execute a shell script.

With EL expression #{Job.getNodeOutput()}, you can obtain the desired content (4000 characters at most and counted backwards) in the output of the shell script run by the Shell node.

Example:

To obtain <name>jack<name1> from a shell script (script name: shell_job1) output, enter the following EL expression:

#{StringUtil.substringBetween(Job.getNodeOutput("shell_job1"),"<name>","<name1>")}

Parameters

Table 1 and Table 2 describe the parameters of the Shell node.

Table 1 Parameters

Parameter

Mandatory

Description

Shell or Script

Yes

You can select Shell statement or Shell script.

  • Shell statement

    In the Shell statement text box, enter the Shell statement to be executed.

  • Shell script

    Select a script to be executed. If no script is available, create and develop a script by referring to Creating a Script and Developing a Shell Script.

    NOTE:

    If you select Shell statement, the DataArts Factory module cannot parse the parameters contained in the Shell statement.

    The execution result of a Shell node cannot be larger than 30 MB. Otherwise, an error is reported.

Host Connection

Yes

Selects the host where a shell script is to be executed.

NOTICE:
  • The maximum number of shell or Python scripts that can run concurrently on the ECS is determined by the value of MaxSessions in the /etc/ssh/sshd_config file on the ECS. Set MaxSessions based on the scheduling frequency of shell or Python scripts.
  • You have the permission to create and execute files in the /tmp directory on the host.
  • Shell and Python scripts are executed in the /tmp directory on an ECS. Ensure that the disk space of the /tmp directory is not used up.

Script Parameter

No

Parameter transferred to the script when the shell script is executed. Parameters are separated by spaces. For example: a b c. The parameter must be referenced by the shell script. Otherwise, the parameter is invalid.

Interactive Input

No

Interactive information (for example, passwords) provided during shell script execution. Interactive parameters are separated by spaces. The shell script reads parameter values in sequence according to the interaction situation.

The following is an example of using the read -p syntax:

read -p "Parameter 1 and parameter 2"Variable 1 Variable 2

Node Name

Yes

Name of the node. It contains a maximum of 128 characters, including letters, digits, hyphens (-), underscores (_), slashes (/), angle brackets (<>), and periods (.).

By default, the node name is the same as that of the selected script. If you want the node name to be different from the script name, disable this function by referring to Disabling Auto Node Name Change.
Table 2 Advanced settings

Parameter

Mandatory

Description

Node Status Polling Interval (s)

Yes

How often the system checks whether the node execution is complete. The value ranges from 1 to 60 seconds.

Max. Node Execution Duration

Yes

Execution timeout interval for the node. If retry is configured and the execution is not complete within the timeout interval, the node will be executed again.

Retry upon Failure

Yes

Whether to re-execute a node if it fails to be executed. Possible values:

  • Yes: The node will be re-executed, and the following parameters must be configured:
    • Retry upon Timeout
    • Maximum Retries
    • Retry Interval (seconds)
  • No: The node will not be re-executed. This is the default setting.
    NOTE:

    If retry is configured for a job node and the timeout duration is configured, the system allows you to retry a node when the node execution times out.

    If a node is not re-executed when it fails upon timeout, you can go to the Default Configuration page to modify this policy.

    Retry upon Timeout is displayed only when Retry upon Failure is set to Yes.

Retry Condition

No

If Retry upon Failure is set to Yes, retry conditions can be set.

Enable Retry Condition and set the return code range.

The shell job can determine whether to retry a failed node based on the return code. You can define the return codes that can be used to determine whether to retry a failed node.

Policy for Handling Subsequent Nodes If the Current Node Fails

Yes

Operation that will be performed if the node fails to be executed. Possible values:

  • Suspend execution plans of the subsequent nodes: stops running subsequent nodes. The job instance status is Failed.
  • End the current job execution plan: Execution of the current job will stop, and the job instance status will become Failed.
  • Go to the next node: ignores the execution failure of the current node. The job instance status is Failure ignored.
  • Suspend the current job execution plan: If the current job instance is in abnormal state, the subsequent nodes of this node and the subsequent job instances that depend on the current job are in waiting state.

Enable Dry Run

No

If you select Dry run, the node will not be executed, and a success message will be returned.

Task Groups

No

Select a task group. If you select a task group, you can control the maximum number of concurrent nodes in the task group in a fine-grained manner in scenarios where a job contains multiple nodes, a data patching task is ongoing, or a job is rerunning.
Parent topic: Node Reference