Updated on 2025-07-10 GMT+08:00

Creating a Connection to a Target Component

To verify the consistency of data stored using big data components, you need to establish connections between MgC and the big data components.

The supported big data components include:

  • Doris
  • HBase
  • ClickHouse
  • Hive Metastore

Procedure

  1. Sign in to the MgC console. In the navigation pane, under Project, select your big data migration project from the drop-down list.
  2. In the navigation pane on the left, choose Migrate > Big Data Verification.
  3. In the Features area, click Preparations.
  4. Choose Connection Management and click Create Connection.

    Figure 1 Creating a connection

  5. Select a big data component and click Next.
  6. Set parameters based on the big data component you selected.

    Table 1 Parameters for creating a connection to Doris

    Parameter

    Configuration

    Connection To

    Select Target.

    Connection Name

    The default name is Doris-4 random characters (including letters and numbers). You can also customize a name.

    MgC Agent

    Select the MgC Agent installed in the source environment.

    Doris Credential

    Select the target Doris credential added to the MgC Agent. For details about how to add credentials, see "Big data - Doris" in Adding Resource Credentials.

    Database IP Address

    Enter the IP address for accessing the target Doris cluster.

    To obtain the address, log in to FusionInsight Manager, choose Cluster > Services > Doris, and check Host Where Leader Locates.

    Database Port

    Enter the port for accessing the target Doris cluster.

    To obtain the port, log in to FusionInsight Manager, choose Cluster > Services > Doris > Configurations, and search for query_port.

    Database Name

    Enter the name of the target Doris database.

    Table 2 Parameters for creating a connection to HBase

    Parameter

    Configuration

    Connection To

    Select Target.

    Connection Name

    The default name is HBase-4 random characters (including letters and numbers). You can also customize a name.

    MgC Agent

    Select the MgC Agent installed in the source environment.

    HBase Credential

    Select the target HBase credential added to the MgC Agent. For details about how to add credentials, see "Big data - HBase" in Adding Resource Credentials.

    Secured Cluster

    Choose whether the cluster is secured.

    ZooKeeper IP Address

    Enter the IP address for connecting to the target ZooKeeper node. You can enter the public or private IP address of the ZooKeeper node.

    ZooKeeper Port

    Enter the port for connecting to the target ZooKeeper node. The default value is 2181.

    HBase Version

    Select the target HBase version.

    Table 3 Parameters for creating a connection to ClickHouse

    Parameter

    Configuration

    Connection To

    Select Target.

    Connection Name

    The default name is ClickHouse-4 random characters (including letters and numbers). You can also customize a name.

    MgC Agent

    Select the MgC Agent installed in the source environment.

    ClickHouse Credential (Optional)

    Select the target ClickHouse credential added to the MgC Agent. For details about how to add credentials, see "Big data - ClickHouse" in Adding Resource Credentials.

    Secured Cluster

    Choose whether the cluster is secured.

    ClickHouse Server IP Address

    Enter the IP address of the target ClickHouse server.

    HTTP Port

    If the ClickHouse cluster is unsecured, enter the HTTP port for communicating with the ClickHouse server. The default value is 8123.

    HTTP SSL/TLS Port

    If the ClickHouse cluster is secured, enter the HTTPS port for communicating with the target ClickHouse server.

    Table 4 Parameters for creating a connection to Hive Metastore

    Parameter

    Configuration

    Connection To

    Select Target.

    Connection Name

    The default name is Hive-Metastore-4 random characters (including letters and numbers). You can also customize a name.

    MgC Agent

    Select the MgC Agent installed in the source environment.

    Secure Connection

    Choose whether to enable secure connection.

    • If Hive Metastore is deployed in an unsecured cluster, do not enable secure connection.
    • If Hive Metastore is deployed in a secured cluster, enable secure connection and provide access credentials. For details about how to obtain and add credentials to MgC, see "Big data - Hive Metastore" in Adding Resource Credentials.

    Hive Version

    Select the target Hive version.

    CAUTION:

    Only version 3.x is available.

    Hive Metastore IP Address

    Enter the IP address for connecting to the Hive Metastore node.

    Hive Metastore Thrift Port

    Enter the port for connecting to the Hive Metastore Thrift service. The default port is 9083.

    Connect to Metadata Database

    During an incremental data verification, querying with Metastore on more than 30,000 partitions may lead to a memory overflow (OOM) since all partition information is loaded into memory. Connecting to the MySQL metadata database can effectively prevent this issue.

    • If you disable this option, the system queries the information of Hive tables and partitions using Hive Metastore.
    • If you enable this option, configure the MySQL database information. The system will query the information of Hive tables and partitions through the MySQL database. You need to set the following parameters:
      • Metadata Database Type: Only MySQL is supported.
      • MySQL Credential: Select the credential for accessing the MySQL database. You need to add the credential to the MgC Agent and synchronize it to MgC. For details, see Adding Resource Credentials.
      • MySQL Node IP Address: Enter the IP address of the MySQL database server.
      • MySQL Port: Enter the port of the MySQL database service.
      • Database Name: Enter the name of the database that stores the Hive table metadata.
    NOTE:

    Ensure that the entered MySQL credential, node IP address, service port, and database name match the MySQL database used by Hive. Otherwise, data verification will fail.

  7. Click Test. MgC verifies whether the component can be connected using the information you provided. If the test is successful, the connection can be set up.
  8. After the connection test is successful, click Confirm. The connection is set up.
  9. On the Connection Management page, view the created connection and its basic information. In the Operation column, click Modify to modify the connection settings.