Help Center > > User Guide> MRS Cluster Component Operation Guide> Using HBase> Enabling the Cross-Cluster Copy Function

Enabling the Cross-Cluster Copy Function

Updated at: Apr 28, 2020 GMT+08:00

Enable the cross-cluster copy function.

Scenario

DistCp is used to back up the data stored on HDFS from a cluster to another cluster. DistCp depends on the cross-cluster copy function, which is disabled by default and needs to be enabled in both clusters.

This section describes how to modify parameters on MRS to enable the cross-cluster copy function.

Impact on the System

Yarn needs to be restarted to enable the cross-cluster copy function and cannot be accessed during the restart.

Prerequisites

The hadoop.rpc.protection parameter of the two clusters must be set to the same data transmission mode, which can be privacy (encryption enabled) or authentication (encryption disabled).

  • For versions later than MRS 2.0.1, choose Components > HDFS > Service Configuration on the cluster details page. Set Type to All, and search for hadoop.rpc.protection.
  • For MRS 1.8.10 or earlier, log in to MRS Manager. Choose Services > HDFS > Service Configuration on the cluster details page. Set Type to All, and search for hadoop.rpc.protection.

Procedure

  1. Log in to MRS cluster details page of a cluster, click Components.

    • If the Components tab is not displayed on the cluster details page, perform IAM user synchronization first. (On the Dashboard tab page of the cluster details page, click on the right side of IAM User Sync to synchronize IAM users.)
    • For MRS 1.8.10 or earlier, log in to MRS Manager. For details, see Accessing MRS Manager. Then, choose Services.

  2. Choose Yarn > Service Configuration and set Type to All.
  3. In the navigation tree, choose Yarn > Distcp.
  4. Set dfs.namenode.rpc-address.haclusterX.remotenn1 to the service IP address and RPC port number of one NameNode instance of the peer cluster, and set dfs.namenode.rpc-address.haclusterX.remotenn2 to the service IP address and RPC port number of the other NameNode instance of the peer cluster.

    • For versions later than MRS 2.0.1, choose Components > HDFS > Instances on the cluster details page to obtain the service IP address of the NameNode instance.
    • For MRS 2.0.1 or later, log in to MRS Manager. Choose Services > HDFS > Instance to obtain the service IP address of the NameNode instance.

    dfs.namenode.rpc-address.haclusterX.remotenn1 and dfs.namenode.rpc-address.haclusterX.remotenn2 do not distinguish active and standby NameNode instances. The default NameNode RPC port number is 9820 and cannot be modified on MRS Manager.

    Examples of the modified parameter values: 10.1.1.1:9820 and 10.1.1.2:9820.

  5. Click Save Configuration, select Restart the role instance, and click OK to start the Yarn service.

    After the system displays Operation succeeded, click Finish. The Yarn service is successfully started.

  6. Log in to MRS cluster details page of the other cluster and repeat the preceding steps.

Did you find this page helpful?

Submit successfully!

Thank you for your feedback. Your feedback helps make our documentation better.

Failed to submit the feedback. Please try again later.

Which of the following issues have you encountered?







Please complete at least one feedback item.

Content most length 200 character

Content is empty.

OK Cancel