Referencing Parameters in Scripts and Jobs

Environment Variable

Variables and constants can be defined in environment variables. Environment variables take effect in current workspace.

• The value of a variable (such as workspace name) varies depending on the workspace. When exporting a variable from a workspace and import it to another workspace, you must reconfigure its value.
• The value of a constant in different workspaces is the same. When importing a constant to another workspace, you do not need to reconfigure its value.

Figure 1 Environment Variable

The specific application is as follows:

An environment variable has been added. The parameter name is sdqw and the parameter value is wqewqewqe.

1. Open a created job and drag a Create OBS node from the node library to the canvas.
2. On the Node Properties tab page, configure the node properties.
   Figure 2 Create OBS
   

3. Click Save and then Monitor to monitor the running status of the job.

Configuring Job Parameters

Script Parameter

• Script parameters take effect in current script and it can be used in the following ways.
  • For SQL scripts, you can directly enter parameters in the script editor (not supported for Flink SQL scripts). During job scheduling, you can assign values to parameters through node attributes, as shown in 2.
  • For Shell scripts, you can enter a parameter and an interactive parameter in the upper part of the editor to transfer the parameters.
  • Python scripts support parameter transfer.
  • For SQL scripts, you can directly enter parameters in the script editor (not supported for Flink SQL scripts). When executing a script independently, you can configure parameters in the lower part of the editor shown in Figure 5.
    Figure 5 Configuring script parameters when executing a script independently
    

1. Developing a Python Script During script development, the script expression must contain variables. For example, if the variable in the SQL statement is DATE, set this parameter to ${DATE} in the script. In the job parameter configuration, you can compile the statement expression of the script parameter Date in 2.
   On the script development page, enter development statements in the editor, as shown in the following figure.

   INSERT INTO B FROM (SELECT * FROM A WHERE DATE = ${DATE})

   Figure 6 Developing a script
   

   After the dws_030903 script is compiled, save and submit the latest version of the script.

2. Develop batch jobs. When developing a job, you need to configure node attribute parameters.

   In the left navigation pane of DataArts Factory, choose Data Development > Develop Job.

   Figure 7 Configuring script parameters when the script is executed by job scheduling
   
   NOTE:

   • If the associated SQL script uses a parameter, the parameter name is displayed (DATA for example). Set the parameter value in the text box next to the parameter name. The parameter value can be an EL expression.
   • If the associated SQL script or script parameters change, you can click to synchronize the changes or click to edit the changes.
   • All nodes involving scripts, such as SQL scripts, shell scripts, and Python scripts, can use this method to reference script variables.

Simple Variable Set

The simple variable set provides a series of customized variables. Customized parameters are automatically replaced with specific values based on the service date, plan time, and parameter value format of task scheduling. In this way, parameters can be dynamically replaced during task scheduling. For details about the simple variable set, see Simple Variable Set.