Help Center/ Cloud Data Migration/ Best Practices/ Migration of IoV Big Data to the Lake Without Loss/ Restoring the HBase Table on MRS
Updated on 2024-08-30 GMT+08:00
View PDF
Share

Restoring the HBase Table on MRS

After the CDH HBase table directories are migrated to MRS HBase, you can run commands to restore the directories. For data that may change, create snapshots to ensure that the data remains unchanged, and then migrate and restore the data.

Constraints

This solution supports only data migration to MRS 1.x clusters. In MRS 2.x and later versions, HBase tables cannot be rebuilt by running HBase repair commands.

If the target cluster version is 2.x or later, the HBase repair command is no longer supported, and the HBase data directory migration cannot be implemented.

Running Commands to Restore the Data Remaining Unchanged

For example, to restore the /hbase/data/default/table_20180811 table, perform the following operations:
  1. Access the node where MRS Client is located, for example, master1.
  2. Run the following command to switch to user omm:

    su – omm

  3. Run the following command to load environment variables:

    source /opt/client/bigdata_env

  4. Run the following command to change the directory permission:

    hdfs dfs –chown omm:hadoop –R /hbase/data/default/table_20180811

    • omm:hadoop: Indicates the username. Replace it with the actual username.
    • /hbase/data/default/table_20180811: Indicates the path of the table.
  5. Run the following command to restore metadata:

    hbase hbck –fixMeta table_20180811

  6. Run the command to restore regions:

    hbase hbck –fixAssignments table_20180811

  7. If the message "Status: OK" is displayed, the table is restored successfully.

Using Snapshots to Migrate and Restore the Data that May Change

  1. Run the following command in the HBase shell of the source CDH cluster:

    flush <table name>

  2. Run the following command in the HBase shell of the source CDH cluster:

    compact <table name>

  3. If the Snap function is not enabled in the table, run the following command to enable the function:

    hadoop dfsadmin -allowSnapshot $path

  4. Run the following commands to create an HDFS snapshot named s0:

    hdfs dfs -createSnapshot <snapshotDir> [s0]

    hdfs dfs -createSnapshot test

  5. CDM copies files to MRS using the HDFS snapshot. Configure the migration job on CDM as follows:
    • Source Directory/File: /hbase/data/default/src_test/.snapshot/s0
    • Write Directory: /hbase/data/default/Table name
  6. Run the fixMeta and fixAssignments commands to restore the table. For details, see Running Commands to Restore the Data Remaining Unchanged.
  7. Run the following command to delete the snapshot from the CDH cluster:

    hdfs dfs -deleteSnapshot <snapshotDir> s0

Rectifying the Fault That Occurs When Restoring a Table

  1. After the fixMeta command is executed, the error message "xx inconsistent" is displayed.

    The fixMeta command is used to check the consistency of metadata between HDFS and HBase. This is a normal situation. Proceed to run the fixAssignments command.

  2. After the fixAssignments command is executed, the error message "xx inconsistent" is displayed.

    The fixAssignments command is used to restore all regions. Sometimes, some regions go online slowly. You can run the following command to check whether the HBase table is successfully restored:

    hbase hbck Table name

    If the message "Status: OK" is displayed, the HBase table is restored successfully.

  3. After the fixAssignments command is executed, an error message is displayed, indicating that multiple regions have the same startkey and some regions overlap.

    Run the following command:

    hbase hbck –fixHdfsOverlaps Table name

    Then run the fixMeta and fixAssignments commands.