Configuring Cascading Authorization for Hive Tables

Scenarios

In traditional authorization mode, permissions on Hive tables are independent of permissions on the underlying storage source (HDFS/OBS). To access a Hive table, a user must obtain permissions for the table and also apply for permissions for the corresponding storage source path. This two-step authorization process is complex and error-prone, significantly increasing O&M costs. Furthermore, when the storage path of a Hive table changes, storage source permissions must be manually reconfigured. Otherwise, the authorization becomes invalid, impacting service continuity.

Ranger cascading authorization is an efficient authorization solution designed for storage-compute decoupled architectures. It automatically associates Hive table permissions with storage source permissions, ensuring they take effect simultaneously.

• Administrators only need to create an authorization policy for the Hive database or table on the Ranger page. The system automatically identifies the corresponding storage source (HDFS/OBS) path and generates fine-grained access permissions for it. Users do not need to be aware of the underlying storage path or undergo a two-step authorization process.
• For an authorized database or table, if the storage source path changes, the system automatically associates the new storage source and updates the corresponding permissions through a periodic synchronization mechanism, ensuring the authorization remains valid.

Prerequisites

This section applies only to MRS 3.3.0 or later clusters. Before using this feature, ensure that the following prerequisites are met:

• The AccessLabel function must be enabled for the OBS file system connected to the MRS cluster. If this function is not enabled, enable it manually. For details, contact OBS O&M personnel.
• To ensure successful OBS authorization for a table, the following prerequisites must be met when you use OBS as the storage source:
  • The Guardian service must have been installed in the MRS cluster.
  • Tables stored in OBS can be authorized only to user groups.
  • In versions earlier than MRS 3.5.0, OBS cascading authorization is supported only for clusters with Kerberos authentication enabled.

Notes and Constraints

• Cascading authorization is not supported for views.
• Cascading authorization can be performed only on databases and tables, not on partitions. If a partition path is outside the table path, you must manually authorize the partition path.
• Cascading authorization is not supported for Deny Conditions in Hive Ranger policies. That is, a Deny Conditions policy restricts only table permissions and cannot generate corresponding access permissions for the HDFS/OBS storage source.
• You cannot create a Hive Ranger policy where both the database and table fields are set to wildcard characters (*).
• HDFS Ranger policies take precedence over HDFS/OBS storage source permissions generated by cascading authorization. If an HDFS Ranger policy has already been configured for a table's HDFS storage source, the cascading permissions will not take effect.
• If cascading authorization is enabled for a table that uses the OBS storage source, you cannot perform ALTER operations on it. To execute this operation, you must grant Read and Write permissions to the corresponding user group on the parent directory of the OBS table path. User group names can contain up to 52 characters, including letters (A-Z, a-z), digits (0-9), underscores (_), and number signs (#). Otherwise, the policy will fail to be added. For how to modify user group information, see Managing MRS Cluster User Groups.
• In MRS 3.6.0-LTS and later versions, Hive cascading authorization can be performed only on the OBS bucket or parallel file system that belongs to the current cluster. It cannot be performed on the OBS buckets or parallel file systems of other clusters.
• In MRS 3.6.0-LTS and later versions, you can configure a cascading authorization blocklist for OBS parallel file systems. When a blocklist is configured, cascading authorization skips the OBS paths specified in the blocklist and authorizes only Hive metadata.

  Log in to FusionInsight Manager, choose Cluster > Services > Ranger, click Configurations, and search for the ranger.ext.authorization.cascade.blacklist parameter. Set its value to obs://OBS bucket name or parallel file system name/. Note that you can configure only OBS paths, wildcards are not supported, and you must use commas (,) to separate multiple entries. A maximum of 50 OBS paths can be configured.

  For example:

  The path of table tblA is obs://obs-test/tblA, and the configured cascading authorization blocklist is obs://obs-test/, obs://obs-test1/.

  When authorizing the tblA table, only the Hive metadata will be authorized, and the path obs:///obs-test/tblA will not be authorized.

  Figure 1 Configuring the cascading authorization blocklist
  

Enabling Cascading Authorization

1. Log in to FusionInsight Manager, choose Cluster > Services > Ranger, and click Configurations.
2. Search for the ranger.ext.authorization.cascade.enable parameter and set it to true.
3. Click Save.
4. Click Instance and select all RangerAdmin instances. Click More and select Restart Instance. Enter the password, and click OK to restart all RangerAdmin instances.
5. Wait until the instance restarts successfully and its status returns to normal.

Connecting to the HDFS Storage Source

The HDFS storage source does not need to be configured.

Connecting to the OBS Storage Source

• Setting the location to an OBS path when creating a table
  1. Ensure that storage and compute decoupling has been configured. For details, see section "Configuring Storage-Compute Decoupling for an MRS Cluster".
  2. Log in to the Ranger web UI as the Ranger administrator. On the homepage, click OBS in the EXTERNAL AUTHORIZATION area, click Add New Policy, and assign Read and Write permissions for the OBS storage path to the user group that contains the target user. For details, see Adding a Ranger Access Permission Policy for OBS Files.

     For example, assign the Read and Write permissions on the obs://obs-test/test/ directory to the hgroup1 user group, as shown in the following figure.

     

  3. On the home page, click the component plug-in name Hive in the HADOOP SQL area. On the Access page, click Add New Policy to add a URL policy that assigns the Read and Write permissions on OBS storage paths to the user group to which the corresponding user belongs. For details, see Adding a Ranger Access Permission Policy for Hive.

     For example, create the hive_url_policy URL policy for the hgroup1 user group and assign the Read and Write permissions on the obs://obs-test/test/ directory to the user group, as shown in the following figure.

     

  4. Log in to the beeline client and set Location to the OBS file system path when creating a table.

     cd Client installation directory

     kinit Component operation user

     beeline

     For example, to create a table named test whose Location is obs://obs-test/test/Database name/Table name, run the following command:

     create table test(name string) location "obs://obs-test/test/Database name/Table name";

• Interconnecting Hive with OBS through Metastore
  1. Ensure that storage and compute decoupling has been configured. For details, see section "Configuring Storage-Compute Decoupling for an MRS Cluster".
  2. Log in to FusionInsight Manager, choose Cluster > Services > Hive, and click Configurations.
  3. Search for hive.metastore.warehouse.dir in the search box and change the parameter value to an OBS path, for example, obs://hivetest/user/hive/warehouse/. hivetest indicates the OBS file system name.
     Figure 2 hive.metastore.warehouse.dir configuration
     
  4. Save the configuration, choose Cluster > Services, and restart the Hive service in the service list.
  5. Update the client configuration file.
     1. Log in to the node where the Hive client is located and run the following command to modify hivemetastore-site.xml in the Hive client configuration file directory:

        vi Client installation directory/Hive/config/hivemetastore-site.xml

     2. Change the value of hive.metastore.warehouse.dir to the corresponding OBS path, for example, obs://hivetest/user/hive/warehouse/.

        

     3. Change the value of hive.metastore.warehouse.dir of hivemetastore-site.xml in the HCatalog client configuration file directory to the corresponding OBS path, for example, obs://hivetest/user/hive/warehouse/.

        vi Client installation directory/Hive/HCatalog/conf/hivemetastore-site.xml

     4. Log in to the Ranger web UI as the Ranger administrator. On the homepage, click OBS in the EXTERNAL AUTHORIZATION area, click Add New Policy, and assign Read and Write permissions for the OBS storage path to the user group that contains the target user.

        For example, assign the Read and Write permissions on the obs://hivetest/user/hive/warehouse/ directory to the hgroup1 user group:

        

     5. Choose Settings > Roles, click Add New Role, and create a role whose Role Name is hive.

        

  6. Go to the Hive Beeline CLI, create a table, and ensure that the location is an OBS path.

     cd Client installation directory

     kinit Component operation user

     beeline

     create table test(name string);

     desc formatted test;

     If a database's location points to HDFS, any table created within that database (without an explicitly specified location) points to HDFS by default. To change this default behavior, modify the database location to point to OBS instead.

     1. Query the location of the database.

        show create database obs_test;

        

     2. Modify the database location.

        alter database obs_test set location 'obs://test1/';

        Run the following command to verify whether the database location points to OBS:

        show create database obs_test;

        

     3. Modify the table location.

        alter table user_info set location 'obs://test1/';

        If the table contains data, migrate the original data file to the new location.

Helpful Links

• For details about the priorities of different Ranger policies, see Condition Priorities of the Ranger Permission Policy.
• To view information about permission objects in Ranger, such as users, user groups, and roles, see Viewing Ranger User Permission Synchronization Information.
• You can create new users for an MRS cluster in MRS Manager. For details, see Creating an MRS Cluster User.