Help Center/ DataArts Studio/ User Guide/ Management Center/ Typical Scenarios for Using Management Center/ Creating a Connection Between DataArts Studio and a MySQL Database
Updated on 2024-10-28 GMT+08:00
View PDF
Share

Creating a Connection Between DataArts Studio and a MySQL Database

This section describes how to create a MySQL connection between DataArts Studio and the data lake base.

Prerequisites

  • You have created a data lake to connect, for example, a database or cloud service supported by DataArts Studio.
    • Before creating a DWS data connection, ensure that you have created a cluster in DWS and have the permissions required to view Key Management Service (KMS) keys.
    • Before creating an MRS connection such as an MRS HBase or MRS Hive connection, ensure that you have purchased an MRS cluster whose Kerberos encryption type is aes256-sha1,aes128-sha1, and that the cluster contains required components.
  • The data lake to connect communicates with the DataArts Studio instance properly.
    • If the data lake is an on-premises database, a public network or a dedicated connection is required. Ensure that the host where the data source is located can access the public network and the port has been enabled in the firewall rule.
    • If the data lake is a cloud service (such as DWS and MRS), the following requirements must be met for network interconnection:
      • If the CDM cluster in the DataArts Studio instance and the cloud service are in different regions, a public network or a dedicated connection is required.
      • If the CDM cluster in the DataArts Studio instance and the cloud service are in the same region, VPC, subnet, and security group, they can communicate with each other by default. If they are in the same VPC but in different subnets or security groups, you must configure routing rules and security group rules. For details about how to configure routing rules, see Configuring Routing Rules. For details about how to configure security group rules, see Configuring Security Group Rules.
      • The cloud service instance and the DataArts Studio workspace belong to the same enterprise project. If they do not, you can modify the enterprise project of the workspace.
  • If the enterprise mode is used, pay attention to the following points:
    In enterprise mode, the development environment and production environment need to be distinguished. Therefore, you need to prepare two sets of data lake services for the production environment and development environment to isolate the development environment from the production environment.
    • If two clusters are used for clustered data sources, such as MRS, GaussDB(DWS), RDS, MySQL, Oracle, DIS, and ECS, you can create data connections in Management Center to distinguish data lake services in the development environment from those in the production environment. The data lake is automatically switched during development and production. Therefore, you need to prepare two sets of data lake services. The versions, specifications, components, regions, VPCs, subnets, and related configurations of the two sets of data lake services must be the same. For details on how to create data connections, see Creating a DataArts Studio Data Connection.
    • For serverless services (such as DLI), DataArts Studio configures the mapping between data lake services in the production environment and development environment through environment isolation in the management center. The corresponding data lake is automatically switched during the development and production processes. Therefore, you need to prepare two sets of queues and database resources in the serverless data lake service and distinguish them by name suffix. For details, see Configuring Environment Isolation for a DataArts Studio Workspace in Enterprise Mode.
    • For GaussDB(DWS), MRS Hive, and MRS Spark, if you select the same cluster when creating a data connection, you must configure database mapping on the Configure Data Source Resource Mapping page to isolate the development and production environments. For details, see DB configuration.
    • Offline processing migration jobs are not supported in enterprise mode.

    For example, if your data lake service is an MRS cluster, you need to prepare two MRS clusters with the same version, specifications, components, region, VPC, and subnet. If some configurations of an MRS cluster are modified, you also need to synchronize the modifications to the other MRS cluster.

Creating a Data Connection

  1. Log in to the DataArts Studio console by following the instructions in Accessing the DataArts Studio Instance Console.
  2. On the DataArts Studio console, locate a workspace and click Management Center.
  3. On the displayed Manage Data Connections page, click Create Data Connection.

    Figure 1 Creating a data connection

  4. On the Manage Data Connections page, click Create Data Connection. On the displayed page, select RDS for Data Connection Type and set other parameters based on the descriptions in Table 1.

    • You are not advised to select MySQL (pending offline) for Data Connection Type. Instead, You are advised to select RDS.
    • RDS data connections depend on OBS. If OBS is unavailable in the same region as DataArts Studio, RDS data connections are not supported.
    Figure 2 RDS connection parameters

    Table 1 RDS connection

    Parameter

    Mandatory

    Description

    Data Connection Type

    Yes

    RDS is selected by default and cannot be changed.

    Name

    Yes

    Name of the data connection to create. Data connection names can contain a maximum of 100 characters. They can contain only letters, digits, underscores (_), and hyphens (-).

    Tag

    No
    Attribute of the data connection to create. Tags make management easier.
    NOTE:

    The tag name can contain only letters, digits, and underscores (_) and cannot start with an underscore (_) or contain more than 100 characters.

    Applicable Modules

    Yes

    Select the modules for which this connection is available.

    NOTE:
    • When the data migration job feature is enabled, you can select the DataArts Migration module. Then you can select this data connection when creating a data migration job in DataArts Factory.
    • You can use offline processing migration jobs only after apply for the trustlist membership. To use this feature, contact customer service or technical support.

    Basic and Network Connectivity Configuration

    IP Address or Domain Name

    Yes

    Address for accessing the relational database data source. The value can be an IP address or a domain name.

    If you choose to manually enter an IP address or domain name, you must enter an internal IP address and a port that is accessible to the network segment of the resource group. Otherwise, the network is disconnected.

    • If the data source is RDS or GaussDB, you can obtain the address on the management console by performing the following operations:
      1. Log in to the management console of the corresponding cloud service using the account you have obtained.
      2. In the left navigation pane, choose Instances.
      3. Click the name of an instance to enter the basic information page. In the Connection Information area, you can obtain the private IP address, domain name, and port number.
      NOTE:

      Only the GaussDB data source supports multiple domain names. Use commas (,) to separate multiple domain names.

    • If the data source is MySQL, PostgreSQL, or DM, you can obtain the access address from the database administrator.

    Port

    Yes

    Port for accessing the relational database.

    • If the data source is RDS or GaussDB, you can obtain the address on the management console by performing the following operations:
      1. Log in to the management console of the corresponding cloud service using the account you have obtained.
      2. In the left navigation pane, choose Instances.
      3. Click the name of an instance to enter the basic information page. In the Connection Information area, you can obtain the private IP address, domain name, and port number.
      NOTE:

      Only the GaussDB data source supports multiple domain names. Use commas (,) to separate multiple domain names.

    • If the data source is MySQL, PostgreSQL, or DM, you can obtain the access address from the database administrator.

    KMS Key

    Yes
    KMS key used to encrypt and decrypt data source authentication information. Select a default or custom key.
    NOTE:

    When you use KMS for encryption through DataArts Studio or KPS for the first time, the default key dlf/default or kps/default is automatically generated. For more information about default keys, see What Is a Default Master Key?.

    Agent

    Yes

    RDS is not a fully managed service and cannot be directly connected to DataArts Studio. A CDM cluster can provide an agent for DataArts Studio to communicate with non-fully-managed services. Therefore, you need to select a CDM cluster when creating an RDS data connection. If no CDM cluster is available, create one first.

    As a network proxy, the CDM cluster must be able to communicate with RDS. To ensure network connectivity, the CDM cluster must be in the same region and AZ and use the same VPC and subnet as RDS. The security group rule must also allow the CDM cluster to communicate with RDS.

    NOTE:

    If a CDM cluster functions as the agent for a data connection in Management Center, the cluster supports a maximum of 200 concurrent active threads. If multiple data connections share an agent, a maximum of 200 SQL, Shell, and Python scripts submitted through the connections can run concurrently. Excess tasks will be queued. You are advised to plan multiple agents based on the workload.

    SSL Encryption

    No

    Whether to enable SSL encrypted transmission.

    Data Source Driver Configuration

    Driver Name

    Yes

    The name of the driver. The following values are available:

    • com.mysql.jdbc.Driver: Select this driver name for RDS for MySQL or MySQL.
    • org.postgresql.Driver: Select this driver name for RDS for PostgreSQL or PostgreSQL.
    • com.microsoft.sqlserver.jdbc.SQLServerDriver: Select this driver name for RDS for SQL Server.
    • dm.jdbc.driver.DmDriver: Select this driver name for the Dameng database.
    • com.huawei.opengauss.jdbc.Driver: Select this driver name for RDS for GaussDB.

    Driver file source

    Yes

    Select the source of the driver file.

    Driver File Path

    Yes

    It specifies the OBS path where the driver file is located. You need to download a .jar driver file from the corresponding official website and upload it to OBS.

    NOTE:
    • The OBS path of the driver file cannot contain Chinese characters.
    • To update the driver, you must restart the CDM cluster in DataArts Migration and then edit the data connection to upload the driver.

    Data Source Authentication and Other Function Configuration

    Username

    Yes

    The username of the database. The username is required for creating a cluster.

    Password

    Yes

    The password for accessing the database. The password is required for creating a cluster.

  5. Click Test to test connectivity of the data connection. If the test fails, the data connection fails to be created.
  6. After the test is successful, click OK to create the data connection.

Reference

  1. What Are the Precautions for Creating an RDS Data Connection?

    When creating an RDS data connection, you need to bind an agent provided by the CDM cluster. Currently, a version of the CDM cluster earlier than 1.8.6 is not supported.