Developing a Python Script

Preparing the Environment

• An ECS named ecs-dgc has been created.
  

  In this example, the ECS uses the CentOS 8.0 64bit with ARM (40 GB) public image and the Python environment. You can log in to the ECS and run the python command to check the Python environment.

  

• You have enabled the DataArts Migration incremental package and created a CDM cluster named cdm-dlfpyhthon. The cluster provides an agent for the DataArts Factory module to communicate with the ECS.
• Ensure that the ECS can communicate with the CDM cluster, which depends on the following conditions:
  • If the CDM cluster and the ECS are in the same region, VPC, subnet, and security group, they can communicate with each other by default. If they are in the same VPC but in different subnets or security groups, you must configure routing rules and security group rules. For details about how to configure routing rules, see Adding Routes in Virtual Private Cloud (VPC) Usage Guide. For details about how to configure security group rules, see Security Group > Adding a Security Group Rule in Virtual Private Cloud (VPC) Usage Guide.
  • If the CDM cluster and the ECS are in different regions, a public network or a dedicated connection is required for enabling communication between the CDM cluster and the cloud service. If the Internet is used for communication, ensure that an EIP has been bound to the CDM cluster, the host where the data source is located can access the Internet, and the port has been enabled in the firewall rules.
  • The ECS and the CDM cluster belong to the same enterprise project. If they do not, you can modify the enterprise project of the workspace.

Constraints

• Python scripts do not support script parameters or job parameters.

Creating an ECS Data Connection

Before developing a Python script, you need to create a connection to the ECS.

1. On the DataArts Studio console, locate a workspace and click Management Center.
   Figure 1 Management Center
   

2. In the navigation pane, choose Manage Data Connections.
   Figure 2 Creating a data connection
   

3. Click Create Data Connection.
   Figure 3 Creating a data connection
   

4. Configure parameters by referring to Table 1 and create a data connection named python_test.

   Table 1 Host connection parameters

   Parameter	Mandatory	Description
   Data Connection Name	Yes	Name of the host connection. The value can contain only letters, digits, hyphens (-), and underscores (_).
   Host Address	Yes	IP address of the host. For details, see section "Viewing Details About an ECS" in Elastic Cloud Server User Guide.
   Agent	Yes	Agents provided by the CDM cluster.
   Port	Yes	SSH port number of the host.
   Username	Yes	Username of the host
   Login Mode	Yes	Selects the login mode of the host. Key pair Password
   Key pair	Yes	If Key Pair is the login mode of the host, the user needs to obtain the private key file, upload it to OBS, and select an OBS path. This parameter is available only when Login Mode is set to Key Pair. NOTE: The uploaded private key file must be in PEM format, and the uploaded private key file and the public key configured on the host must be in the same key pair.
   Key Pair Password	No	If no password is set for the key pair, you do not need to set this parameter.
   Password	Yes	If the login mode of the host is to use a password, enter a login password.
   Host Connection Description	No	Descriptive information about the host connection.

   

   The key parameters are as follows:

   • Host Address: Enter the IP address of the ECS.
   • Agent: Select the CDM cluster.

5. Click Test to test connectivity of the data connection. If the test fails, the data connection fails to be created.
6. After the test is successful, click OK. The system will create the data connection for you.

Developing a Python Script

1. Choose DataArts Factory > Develop Script and create a Python script named python_test.
2. Edit the Python statement in the editor, select the host connection, and click Submit and Unlock..
3. Click Execute to execute the Python statement.
4. View the script execution result.