Updating the MRS Data Source Configuration

Scenario

For MRS, if the following parameter configurations of the HDFS cluster change, data may fail to be imported to the DWS cluster from the HDFS cluster. Before importing data using the HDFS cluster, you must update the MRS data source configuration.

Prerequisites

You have created an MRS data source connection for the DWS cluster.

Impact on the System

When you are updating an MRS data source connection, the DWS cluster will automatically restart and cannot provide services.

Procedure

1. Log in to the DWS console.
2. Choose Cluster > Cluster List.
3. In the cluster list, click the name of the target cluster. The Cluster Information page is displayed.
4. In the navigation pane, choose Data Source > MRS Data Source.
5. In the MRS data source list, select the MRS data source that you want to update. In the Operation column, click Update Configurations.

   MRS Cluster Status and Configuration Status of the current connection will be updated. During configuration update, you cannot create a connection. The system checks whether the security group rule is correct. If the rule is incorrect, the system rectifies the fault. The following table describes the parameters.

   Table 1 Parameter description

   Parameter	Description
   dfs.client.read.shortcircuit	Specifies whether to enable local read.
   dfs.client.read.shortcircuit.skip.checksum	Specifies whether to skip data verification during the local read.
   dfs.client.block.write.replace-datanode-on-failure.enable	Specifies whether to replace the location storing copies with the new node when data blocks fail to be written to HDFS.
   dfs.encrypt.data.transfer	Specifies whether to enable data encryption. The value true indicates that the channels are encrypted. The channels are not encrypted by default. This parameter is available only for clusters with Kerberos authentication enabled. This parameter is valid only when hadoop.rpc.protection is set to privacy.
   dfs.encrypt.data.transfer.algorithm	Specifies the encryption and decryption algorithm for key transmission. This parameter is valid only when dfs.encrypt.data.transfer is set to true. The default value is 3des, indicating that the 3DES algorithm is used for encryption.
   dfs.encrypt.data.transfer.cipher.suites	Specifies the encryption and decryption algorithm for the transmission of actually stored data. If this parameter is not specified, the cryptographic algorithm specified by dfs.encrypt.data.transfer.algorithm is used for data encryption. The default value is AES/CTR/NoPadding.
   dfs.replication	Specifies the default number of data copies.
   dfs.blocksize	Specifies the default size of a data block.
   hadoop.security.authentication	Specifies the security authentication mode.
   hadoop.rpc.protection	RPC communication protection mode Security mode (Kerberos authentication enabled) authentication: Only authentication is required. integrity: Authentication and consistency check are required. privacy (default): Authentication, consistency check, and encryption are required. Common mode (Kerberos authentication disabled): The default value is authentication.
   dfs.domain.socket.path	Specifies the locally used Domain Socket path.