Help Center> DataArts Studio> User Guide> Management Center> Tutorials> Creating a DWS Connection
Updated on 2024-04-29 GMT+08:00
View PDF

Creating a DWS Connection

This section describes how to create a DWS connection between DataArts Studio and the data lake base.

Prerequisites

  • You have created a data lake to connect, for example, a database or cloud service supported by DataArts Studio.
    • Before creating a DWS data connection, ensure that you have created a cluster in DWS and have the permissions required to view Key Management Service (KMS) keys.
    • Before creating an MRS connection such as an MRS HBase and MRS Hive connection, ensure that you have bought an MRS cluster and selected required components.
  • The data lake to connect communicates with the DataArts Studio instance properly.
    • If the data lake is an on-premises database, a public network or a dedicated connection is required. Ensure that the host where the data source is located can access the public network and the port has been enabled in the firewall rule.
    • If the data lake is a cloud service (such as DWS and MRS), the following requirements must be met for network interconnection:
      • If the CDM cluster in the DataArts Studio instance and the cloud service are in different regions, a public network or a dedicated connection is required.
      • If the CDM cluster in the DataArts Studio instance and the cloud service are in the same region, VPC, subnet, and security group, they can communicate with each other by default. If they are in the same VPC but in different subnets or security groups, you must configure routing rules and security group rules. For details about how to configure routing rules, see configuring routes. For details about how to configure security group rules, see configuring security group rules.
      • The cloud service instance and the DataArts Studio workspace belong to the same enterprise project. If they do not, you can modify the enterprise project of the workspace.
  • If the enterprise mode is used, pay attention to the following points:
    In enterprise mode, the development environment and production environment need to be distinguished. Therefore, you need to prepare two sets of data lake services for the production environment and development environment to isolate the development environment from the production environment.
    • If two clusters are used for clustered data sources, such as MRS, GaussDB(DWS), RDS, MySQL, Oracle, DIS, and ECS, you can create data connections in Management Center to distinguish data lake services in the development environment from those in the production environment. The data lake is automatically switched during development and production. Therefore, you need to prepare two sets of data lake services. The versions, specifications, components, regions, VPCs, subnets, and related configurations of the two sets of data lake services must be the same. For details on how to create data connections, see Creating a Data Connection.
    • For serverless services (such as DLI), DataArts Studio configures the mapping between data lake services in the production environment and development environment through environment isolation in the management center. The corresponding data lake is automatically switched during the development and production processes. Therefore, you need to prepare two sets of queues and database resources in the serverless data lake service and distinguish them by name suffix. For details, see Configuring Environment Isolation for a Workspace in Enterprise Mode.
    • For GaussDB(DWS), MRS Hive, and MRS Spark, if you select the same cluster when creating a data connection, you must configure database mapping on the Configure Data Source Resource Mapping page to isolate the development and production environments. For details, see DB configuration.

    For example, if your data lake service is an MRS cluster, you need to prepare two MRS clusters with the same version, specifications, components, region, VPC, and subnet. If some configurations of an MRS cluster are modified, you also need to synchronize the modifications to the other MRS cluster.

Creating a Data Connection

  1. Log in to the DataArts Studio console by following the instructions in Accessing the DataArts Studio Instance Console.
  2. On the DataArts Studio console, locate a workspace and click Management Center.
  3. On the displayed Manage Data Connections page, click Create Data Connection.

    Figure 1 Creating a data connection

  4. On the Manage Data Connections page, click Create Data Connection. On the displayed page, select DWS for Data Connection Type and set other parameters based on the descriptions in Table 1.

    Figure 2 DWS connection parameters

    Table 1 DWS connection

    Parameter

    Mandatory

    Description

    Data Connection Type

    Yes

    DWS is selected and cannot be changed.

    Name

    Yes

    Name of the data connection to create. Data connection names can contain a maximum of 100 characters. They can contain only letters, digits, underscores (_), and hyphens (-).

    Tag

    No
    Attribute of the data connection to create. Tags make management easier.
    NOTE:

    The tag name can contain only letters, digits, and underscores (_) and cannot start with an underscore (_) or contain more than 100 characters.

    Applicable Modules

    Yes

    Select the modules for which this connection is available.

    All modules are selected by default, which means this connection is available for all the modules that support the data source connected by this connection. For details about the data sources supported by each module, see Data Sources.

    Basic and Network Connectivity Configuration

    SSL Encryption

    Yes

    DWS supports SSL encryption and certificate authentication for communication between the client and server. You can use SSL Connection to set the communication mode. If SSL Connection is enabled, only SSL encryption can be used. If SSL Connection is disabled, both modes can be used. This function is disabled by default.

    Manual

    Yes

    Select either of the following modes:

    • Cluster Name Mode: Select an existing cluster.
    • Connection String Mode: Enter the IP address/domain name and port of the corresponding cluster and enable the communication between the connection's agent (CDM cluster) and the DWS cluster.

    DWS Cluster Name

    Yes

    This parameter is mandatory when Manual is set to Cluster Name Mode.

    Select a DWS cluster from all the DWS clusters with the same project ID and enterprise project.

    IP Address or Domain Name

    Yes

    This parameter is mandatory when Manual is set to Connection String Mode.

    This parameter indicates the address for accessing the cluster database through an internal network. Enter an IP address or domain name. The IP address or domain name is automatically generated during cluster creation. You can obtain them on the management console by performing the following operations:
    1. Log in to the GaussDB(DWS) console.
    2. In the left navigation pane, choose Instances.
    3. Click the name of an instance to enter the basic information page. In the Connection Information area, you can obtain the private IP address, domain name, and port number.

    Port

    Yes

    This parameter is mandatory when Manual is set to Connection String Mode.

    This parameter indicates the database port number specified during the DWS cluster creation. Ensure that you have enabled this port in the security group rule so that the DataArts Studio instance can connect to the database in the DWS cluster through this port.

    KMS Key

    Yes

    KMS key used to encrypt and decrypt the authentication information for the data source

    Agent

    Yes

    Data Warehouse Service (DWS) is not a fully managed service and thus cannot be directly connected to DataArts Studio. A CDM cluster can provide an agent for DataArts Studio to communicate with non-fully-managed services. Therefore, you need to select a CDM cluster when creating a DWS data connection. If no CDM cluster is available, create one first.

    As a network proxy, the CDM cluster must be able to communicate with the DWS cluster. To ensure network connectivity, the CDM cluster must be in the same region, AZ, VPC, and subnet as the DWS cluster. The security group rule must also allow the CDM cluster communicate with the DWS cluster.

    NOTE:

    If a CDM cluster functions as the agent for a data connection in Management Center, the cluster supports a maximum of 200 concurrent active threads. If multiple data connections share an agent, a maximum of 200 SQL, Shell, and Python scripts submitted through the connections can run concurrently. Excess tasks will be queued. You are advised to plan multiple agents based on the workload.

    Data Source Authentication and Other Function Configuration

    Username

    Yes

    The database username, which is specified when the DWS cluster is created.

    Password

    Yes

    The password for accessing the database, which is specified when the DWS cluster is created.

  5. Click Test to test connectivity of the data connection. If the test fails, the data connection fails to be created.
  6. After the test is successful, click OK to create the data connection.

Reference

  1. What should I do if the connection test fails when I enable the SSL connection during the creation of a DWS data connection?
    The failure may be caused by the rights separation function of the DWS cluster. On the DWS console, click the corresponding cluster, choose Security Settings, and disable Rights Separation.
    Figure 3 Disabling Rights Separation for the DWS cluster
  2. Why does a DWS data connection fail to obtain information about databases or tables?

    The possible cause is that the CDM cluster is stopped or a concurrency conflict occurs. You can switch to another agent to temporarily avoid this issue.

Parent topic: Tutorials