Updated on 2024-06-14 GMT+08:00

Managing Connections

Description

Before creating a data migration task, you need to create a connection, so that the cluster can read and write the data source. A migration job requires a source connection and a destination connection. Data sources that support exporting are used as source connections and data sources that support importing are used as destination connections.

The connection parameters you can configure vary according to the data source. This section describes how to create these connections.

Prerequisites

  • A GDS-kafka cluster has been created.
  • The GDS-kafka cluster can communicate with the destination data source.
    • If the destination data source is an on-premises database, you need the Internet or Direct Connect. If the Internet is used for communication, ensure that an EIP has been bound to the GDS-kafka cluster, the security group of GDS-kafka allows outbound traffic from the host where the off-cloud data source is located, the host where the data source is located can access the Internet, and the connection port has been enabled in the firewall rules.
    • If the destination data source is a cloud service, the following requirements must be met for network interconnection:
      • If the GDS-kafka cluster and the cloud service are in different regions, the Internet or a Direct Connect is required for enabling communication between the CDM cluster and the cloud service. If the Internet is used for communication, ensure that an EIP has been bound to the GDS-kafka cluster, the host where the data source is located can access the Internet, and the port has been enabled in the firewall rules.
      • If the GDS-kafka cluster and the cloud service are in the same region, VPC, subnet, and security group, they can communicate with each other by default. If they are in the same VPC but in different subnets or security groups, you must configure routing rules and security group rules. For more information, see Configuring Routes and Security Group Configuration.
      • The cloud service instance and the cluster belong to the same enterprise project. If they do not, you can modify the enterprise project of the workspace.
  • You have obtained the URL, account, and password for accessing the destination data source. The account is granted with the read and write permissions on the data source.

Creating a Connection

  1. Log in to the GaussDB(DWS) console.
  2. In the navigation pane, choose Data > Data Integration > Connections.
  3. Click Create Connection.
  4. Configure connection parameters. For more information, see Connection parameters.

    Table 1 Connection parameters

    Protocol

    Parameter

    Mandatory

    Description

    Kafka

    Connection Name

    Yes

    Connection name, which can be customized.

    Only letters, numbers, underscores (_), and hyphens (-) are allowed.

    Type

    Yes

    Currently, MRS-Kafka, IoT-Kafka, DMS-Kafka, and Default-Kafka are supported. Default-Kafka is an open-source Kafka.

    Service Address

    Yes

    Kafka connection address.

    Format: Domain name + Port number or IP address + Port number

    Topics

    Yes

    List of Kafka topics, which are separated by commas (,).

    Ciphertext Access

    No

    This function must be enabled during Kafka authentication. The SASL_SSL and SASL_PLAINTEXT protocols are supported.

    User

    No

    Username for connecting to Kafka

    Password

    No

    Password for connecting to Kafka.

    SSL Authentication

    No

    Whether the SSL protocol is supported.

    Certificate

    No

    SSL certificate in binary JKS format.

    Certificate Password

    No

    Certificate encryption password.

    Host Configuration

    No

    MRS-Kafka configuration parameter. When you connect to MRS-Kafka in security mode, you need to configure the host file of the VM where Gds-Kafka resides. Therefore, you need to upload the host file to be modified. The file format can only be TXT. The file content is as follows:

    192.168.4.111  node-master1JuQr.mrs-yd8z.com
    192.168.4.204  node-master3mgqy.mrs-yd8z.com
    192.168.4.221  node-master2Ktgg.mrs-yd8z.com

    The information on the left is the IP address of the Kafka broker. If MRS-Kafka and GDS-Kafka are not in the same VPC, replace the IP address with a public IP address. The information on the right is the host name of the broker. You can log in to FusionInsightManage and access the Kafka cluster to obtain the host name corresponding to the broker instance.

    Security mode

    No

    MRS-Kafka configuration parameter. When the security mode is enabled, Kerberos authentication is required.

    Krb5 File

    No

    MRS-Kafka configuration parameter. When the security mode is enabled, you need to upload the krb5 file. This file is the authentication credential of the machine-machine account applied for on FusionInsight Manager of MRS.

    NOTE:

    If MRS-Kafka and GDS-Kafka are not in the same VPC, replace the internal IP address of the broker in the file with the public IP address.

    Keytab File

    No

    MRS-Kafka configuration parameter. When the security mode is enabled, you need to upload the Keytab file. This file is the authentication credential of the machine-machine account applied for on FusionInsight Manager of MRS.

    Account

    No

    MRS-Kafka configuration parameter. It is a machine-machine account applied for on FusionInsight Manager of MRS.

    SSL

    No

    MRS-Kafka configuration parameter. When SSL is enabled, you need to upload the SSL certificate and key.

    Authentication Mechanism

    No

    DMS-Kafka configuration parameter. It indicates the security authentication protocol.

    MySQL

    Connection Name

    Yes

    Connection name, which can be customized.

    Only letters, numbers, underscores (_), and hyphens (-) are allowed.

    Service Address

    Yes

    MySQL connection address.

    Format: Domain name + Port number or IP address + Port number

    User

    Yes

    Username for logging in to the database.

    Password

    Yes

    Password used to log in to the database.

    Database

    Yes

    MySQL database name.

    Oracle

    Connection Name

    Yes

    Connection name, which can be customized.

    Only letters, numbers, underscores (_), and hyphens (-) are allowed.

    Service Address

    Yes

    Oracle connection address.

    Format: Domain name + Port number or IP address + Port number

    User

    Yes

    Username for logging in to the database.

    Password

    Yes

    Password used to log in to the database.

    Database

    Yes

    Oracle database name.

    Schema

    Yes

    Schema name. You can configure one or more schema names and use commas (,) to separate them.

    IoT

    Service Address

    Yes

    Address of the iot-edge-node page.

    Format: Domain name or IP address

    User

    Yes

    Account for logging in to the IoT platform.

    Password

    Yes

    Password for logging in to the IoT platform.

    DWS

    Connection Name

    Yes

    Connection name, which can be customized.

    Only letters, numbers, underscores (_), and hyphens (-) are allowed.

    Service Address

    Yes

    GaussDB(DWS) connection address.

    User

    Yes

    Username for logging in to the database.

    Password

    Yes

    Password used to log in to the database.

    Database

    Yes

    GaussDB(DWS) database name.

    Schema

    Yes

    Name of a schema in the GaussDB(DWS) database.

    DWS Cluster

    Connection Name

    Yes

    Connection name, which can be customized.

    Only letters, numbers, underscores (_), and hyphens (-) are allowed.

    Cluster Name

    Yes

    Name of a GaussDB(DWS) cluster.

    Username

    Yes

    Username of the database corresponding to the GaussDB(DWS) cluster.

    Password

    Yes

    Password of the database corresponding to the GaussDB(DWS) cluster.

    Database

    Yes

    Name of the database corresponding to the GaussDB(DWS) cluster.

    Schema Name

    No

    Schema of the database corresponding to the GaussDB(DWS) cluster.

    MRS

    Connection Name

    Yes

    Connection name, which can be customized.

    Only letters, numbers, underscores (_), and hyphens (-) are allowed.

    Manager Address

    Yes

    Address of the MRS cluster management page.

    Username

    Yes

    Username for logging in to the MRS cluster management page.

    Password

    Yes

    Password for logging in to the MRS cluster management page.

  5. Confirm the information and click OK.

Modifying a Connection

  1. Log in to the GaussDB(DWS) console.
  2. In the navigation pane, choose Data > Data Integration > Connections.
  3. In the Operation column of a connection, click Modify.
  4. In the Modify Connection dialog box, configure Connection Name, Connection Address, Topics, User, and Password.
  5. Confirm the information and click OK.

Deleting a Connection

  1. Log in to the GaussDB(DWS) console.
  2. In the navigation pane, choose Data > Data Integration > Connections.
  3. In the Operation column of a connection, click Delete.
  4. In the displayed dialog box, click OK.