Overview

MgC allows you to verify the consistency of data migrated from various big data computing and storage engines, such as Hive, HBase, Doris, and MaxCompute. Consistency verification ensures data accuracy and reliability and enables to you migrate big data to Huawei Cloud with confidence.

Verification Methods

Full verification: The consistency of all inventory data is verified.
Daily incremental verification: The consistency of incremental data is verified based on the creation or update time. You can choose to verify incremental data for one day or several consecutive days.
Hourly incremental verification: Data consistency is verified based on the creation time or update time multiple times within 24 hours. The verification automatically stops at 00:00 on the next day.
Date-based verification: This method applies only to tables partitioned by date in the year, month, and day format. You can choose to verify consistency of such tables for one day or several consecutive days. Tables that are not partitioned by date are not verified.
Selective verification: This method can be used to verify the consistency of the data within a specified time period. You can only select a period going backward from the current time for verification.

Supported Source and Target Components

Source Component	Target Component
Hive HBase Doris MaxCompute ClickHouse Delta Lake Hudi	Hive DLI MRS (Doris) MRS (Hbase) MRS (ClickHouse) CloudTable (ClickHouse) CloudTable (HBase) Delta Hudi

Verification Methods Available for Each Component

Component	Verification Method
Hive	Full verification Daily incremental verification Hourly incremental verification Date-based verification
DLI
MaxCompute	Full verification Daily incremental verification Hourly incremental verification Date-based verification Sample partition verification
Doris	Full verification Daily incremental verification Hourly incremental verification
HBase	Full verification Selective verification
ClickHouse	Full verification
ApsaraDB for ClickHouse	Full verification
CloudTable (HBase)	Full verification Selective verification
CloudTable (ClickHouse)	Full verification
Delta	Full verification Daily incremental verification Hourly incremental verification Date-based verification
Hudi	Full verification Daily incremental verification Hourly incremental verification Date-based verification