Updated on 2024-10-21 GMT+08:00

Creating a Big Data Service Connection

To verify the consistency of data stored using big data cloud services, you need to establish connections between MgC and the source and target cloud services.

The supported big data cloud services include:

  • Data Lake Insight (DLI)
  • CloudTable (ClickHouse)
  • CloudTable (HBase)

Procedure

  1. Sign in to the MgC console.
  2. In the navigation pane on the left, choose Migrate > Big Data Verification. Select a migration project in the upper left corner of the page.
  3. In the Features area, click Connection Management.
  4. Click Create Connection in the upper right corner of the page.
  5. Select a big data cloud service and click Next.
  6. Configure the connection parameters based on the big data service you selected.

    Table 1 Parameters for creating a connection to DLI

    Parameter

    Configuration

    Connection Source

    Select Target.

    Connection Name

    The default name is DLI-4 random characters (including letters and numbers). You can also customize a name.

    DLI Credential

    Select the credential used for accessing DLI. For details about how to add credentials, see "Big Data - DLI" in Adding Resource Credentials. If the selected credential is the one you currently use to access MgC, you can select This is my MgC credential, and the projects in the region you choose will be listed.

    Region/Project

    Select the region and project where the data to be verified is located.

    Queue

    Enter the name of the DLI queue to be verified. The queue must be a SQL queue.

    Table 2 Parameters required for creating a connection to CloudTable (ClickHouse)

    Parameter

    Configuration

    Connection Source

    Select Target.

    Connection Name

    The default name is CloudTable-ClickHouse4 random characters (including letters and numbers). You can also customize a name.

    CloudTable (ClickHouse) Credential

    Select the credential of the CloudTable (ClickHouse) cluster. For details about how to add credentials, see "Big Data - ClickHouse" in Adding Resource Credentials. When you add the credential, select Username/Password for Authentication and enter the username and password for logging in to the ClickHouse cluster created on CloudTable.

    Database URL

    Enter the URL to access the CloudTable (ClickHouse) database. You can obtain the access address from the basic information of the ClickHouse cluster.

    Table 3 Parameters required for creating a connection to CloudTable (HBase)

    Parameter

    Configuration

    Connection Source

    Select Target.

    Connection Name

    The default name is CloudTable-HBase4 random characters (including letters and numbers). You can also customize a name.

    CloudTable (HBase) Credential

    No credential is required for unsecured clusters.

    Secured Cluster

    Currently, only unsecured clusters (common clusters) are supported.

    Database URL

    Enter the URL to access the CloudTable (HBase) database. You can obtain the access address from thebasic information of the HBase cluster of the HBase cluster. The address displayed after ZK Link (Intranet) is one you need.

  7. Click Test. MgC verifies whether the cloud service can be connected using the configuration information. If the test is successful, the cloud services can be connected.
  8. After the connection test is successful, click Confirm. The cloud service connection is created.
  9. On the Connection Management page, view the created connection and its basic information. In the Operation column, click Modify to modify the connection settings.