Accessing OBS Using HDFS Through Guardian

After Guardian is interconnected with OBS by referring to Disabling Ranger OBS Path Authentication for Guardian or Enabling Ranger OBS Path Authentication for Guardian, you can view and create OBS file directories on the HDFS client.

Prerequisites

If Guardian is connected to OBS by referring to Enabling Ranger OBS Path Authentication for Guardian, ensure that you have the read and write permissions on OBS path in Ranger. For details about how to grant the permissions, see Configuring Ranger Permissions.

Interconnecting HDFS with OBS

Log in to the node on which the HDFS client is installed as a client installation user.
Run the following command to switch to the client installation directory:
```
cd Client installation directory
```
Run the following command to configure environment variables:
```
source bigdata_env
```
If the cluster is enabled with Kerberos authentication, run the following command to perform user authentication. The user must have the read and write permissions on the OBS directory. User authentication is not required for clusters with Kerberos authentication disabled.
```
kinit User performing HDFS operations
```
Access the OBS file system using the HDFS CLI.

The following commands are examples.
- Access the OBS file system.
```
hdfs dfs -ls obs://OBS parallel file system name/path
```
- Create a directory in the OBS file system.
```
hdfs dfs -mkdir obs://OBS parallel file system name/hadoop
```
- Upload the /opt/test.txt file on the client node to the obs://OBS parallel file system name/hadoop directory.
```
hdfs dfs -put /opt/test.txt obs://OBS parallel file system name/hadoop
```
  View the uploaded file in the corresponding OBS file system path.

Changing the Log Level of OBS Client

If a large number of logs are printed in the OBS file system, the read and write performance may be affected. You can adjust the log level of the OBS client as follows:

Go to the hadoop directory.

cd Client installation directory/HDFS/hadoop/etc/hadoop

Edit the file log4j.properties.
```
vi  log4j.properties
```
Add the following OBS log level configuration to the file and save it.
```
log4j.logger.org.apache.hadoop.fs.obs=WARN
log4j.logger.com.obs=WARN
```
Run the following command:
```
tail -4 log4j.properties
```
If the command output shown in Figure 1 is displayed, the log level is successfully changed.

Figure 1 Adding an OBS log level

Configuring Ranger Permissions

Log in to FusionInsight Manager of the MRS cluster.

For details about how to log in to FusionInsight Manager, see Accessing MRS Manager.
Choose System > Permission > User Group. On the displayed page, click Create User Group to create a user group without any roles, for example, obs_hadoop.
Back to FusionInsight Manager and choose System > Permission > User. On the displayed page, click Create User to create a user that is associated only with the obs_hadoop user group, for example, hadoopuser.
Log in to the Ranger management page as the rangeradmin user.
On the home page, click component plug-in name OBS in the EXTERNAL AUTHORIZATION area.
Click Add New Policy and add the Read and Write permissions on the desired OBS paths to the created user group.

The following figure shows the configurations needed for adding the Read and Write permissions on obs://OBS parallel file system name/hadoop to user group obs_hadoop.

Figure 2 Granting the Hadoop user group permissions for reading and writing OBS paths

Before configuring permission policies for OBS paths on Ranger, ensure that the AccessLabel function has been enabled for OBS. If the function is not enabled, manually enable it. For details, contact OBS O&M personnel.