Help Center> MapReduce Service> Component Operation Guide (LTS)> Using HetuEngine> HetuEngine Permission Management> HetuEngine MetaStore-based Permission Control
Updated on 2024-05-29 GMT+08:00
View PDF

HetuEngine MetaStore-based Permission Control

Constraints: This function applies only to Hive data sources.

When multiple HetuEngine clusters are deployed for collaborative computing, the metadata is centrally managed by the management cluster. Data computing is performed in all clusters. The user permission for accessing HetuEngine clusters must be configured in the management cluster. Users who belong to the Hive user group and share the same name are added to all compute instances.

Enabling MetaStore Authentication

  1. Log in to FusionInsight Manager.
  2. If Kerberos authentication is disabled for the cluster (the cluster is in normal mode), add the ranger.usersync.sync.source parameter. If Kerberos authentication is enabled for the cluster (the cluster is in security mode), skip this step. This step is suitable for MRS 3.3.0 or later.

    1. Choose Cluster > Services > Ranger. Click Configurations then All Configurations.
    2. Search for the ranger.usersync.config.expandor parameter, set its name to ranger.usersync.sync.source, set its value to ldap, and save the settings.
    3. On the Dashboard page, click More > Restart Service in the upper right corner, enter the password, and restart Ranger.

  3. Choose Cluster > Services > HetuEngine. Click More and select Disable Ranger.
  4. Choose Cluster > Services > HetuEngine. Click More and select Restart Service.
  5. Restart the compute instance on the HSConsole page. For details, see Restarting HetuEngine Compute Instances.

MetaStore Permission

Similar to Hive, HetuEngine is a data warehouse framework built on Hadoop, providing storage of structured data like SQL.

Permissions in a cluster must be assigned to roles which are bound to users or user groups. Users can obtain permissions only by binding a role or joining a group that is bound with a role.

Permission Management

HetuEngine permission management is performed by the permission system to manage users' operations on the database, ensuring that different users can operate databases independently and securely. A user can operate another user's tables and databases only with the corresponding permissions. Otherwise, operations will be rejected.

HetuEngine permission management integrates the functions of Hive permission management. MetaStore service of Hive and the function of granting permissions on the web page are required to enable the HetuEngine permission management.

  • Granting permissions on the web page: HetuEngine supports only granting permissions on the web page. On Manager, choose System > Permission to add or delete a user, user group, or a role, and to grant or cancel permissions.
  • Obtaining and judging a service: When the DDL and DML commands are received from the client, HetuEngine will obtain the client user's permissions on database information from MetaStore, and check whether the required permissions are included. If the required permissions have been obtained, the user's operations are allowed. If the permissions are not obtained, the user's operation will be rejected. After the MetaStore permissions are checked, ACL permission also needs to be checked on HDFS.

HetuEngine Permission Model

If a user uses HetuEngine to perform SQL query, the user must be granted with permissions of HetuEngine databases and tables (include external tables and views). The complete permission model of HetuEngine consists of the metadata permission and HDFS file permission. Permissions required to use a database or a table are just one type of HetuEngine permission.

  • Metadata permissions

    Metadata permissions are controlled at the metadata level. Similar to traditional relational databases, the HetuEngine database contains the CREATE and SELECT permissions. Tables and columns contain the SELECT, INSERT, UPDATE, and DELETE permissions. HetuEngine also supports the owner permission OWNERSHIP and cluster administrator permission ADMIN.

  • Data file permissions (that is, HDFS file permissions)

    HetuEngine database and table files are stored in HDFS. The created databases or tables are saved in the /user/hive/warehouse directory of HDFS by default. The system automatically creates subdirectories named after database names and database table names. To access a database or a table, the corresponding file permissions (READ, WRITE, and EXECUTE) on HDFS are required.

To perform various operations on HetuEngine databases or tables, you need to associate the metadata permission and the HDFS file permission. For example, to query HetuEngine data tables, you need to associate the metadata permission SELECT with the READ and EXECUTE permissions on HDFS files.

To use the management function of FusionInsight Manager GUI to manage the permissions of HetuEngine databases and tables, you only need to configure the metadata permission, and the system will automatically associate and configure the HDFS file permission. In this way, operations on the interface are simplified, improving efficiency.

HetuEngine Application Scenarios and Related Permissions

A user needs to join in the Hive group if a database is created using the HetuEngine service, and role authorization is not required. Users have all permissions on the databases or tables created by themselves in Hive or HDFS. They can create tables, select, delete, insert, or update data, and grant permissions to other users to allow them to access the tables and corresponding HDFS directories and files.

A user can access the tables or database only with permissions. Permissions required for the user vary depending on different HetuEngine scenarios.

Table 1 Typical HetuEngine scenarios and required permissions

Scenario

Required Permission

Using HetuEngine tables, columns, or databases

Permissions required in different scenarios are as follows:

  • To create a table, the CREATE permission is required.
  • To query data, the SELECT permission is required.
  • To insert data, the INSERT permission is required.

In some special HetuEngine scenarios, other permissions must be configured separately.

Table 2 Typical HetuEngine authentication scenarios and required permissions

Scenario

Required Permission

Creating HetuEngine databases, tables, and foreign tables, or adding partitions to created tables or foreign tables when data files specified by Hive users are saved to other HDFS directories except /user/hive/warehouse.

The directory must exist, the client user must be the owner of the directory, and the user must have the READ, WRITE, and EXECUTE permissions on the directory. The user must have the READ and EXECUTE permissions of all the upper-layer directories of the directory.

Performing operations on all databases and tables in Hive

The user must be added to the supergroup user group, and be assigned the ADMIN permission.

Configuring Permissions for Tables, Columns, and Databases

After MetaStore authentication is enabled, if a user needs to access HetuEngine tables or databases created by other users, the user needs to be granted with related permissions. HetuEngine supports permission control based on columns for strict permission control. If a user needs to access some columns in tables created by other users, the user must be granted the permission for columns.

  • Any permission for a table in the database is automatically associated with the HDFS permission for the database directory to facilitate permission management. When any permission for a table is canceled, the system does not automatically cancel the HDFS permission for the database directory to ensure performance. In this case, users can only log in to the database and view table names.
  • When the query permission on a database is added to or deleted from a role, the query permission on tables in the database is automatically added to or deleted from the role. This mechanism is inherited from Hive.
  • In HetuEngine, the name of a column of the struct type data cannot contain special characters, that is, characters other than letters, digits, and underscores (_). If the column name of the struct data type contains special characters, the column cannot be displayed on the FusionInsight Manager console when you grant permissions to roles on the Role page.

Procedure

  1. Log in to FusionInsight Manager.
  2. Choose System > Permission > Role.
  3. Click Create Role, and set Role Name and Description.
  4. In the Configure Resource Permission area, choose Name of the desired cluster > Hive and set role permissions. For details, see Table 3.

    • Hive Admin Privilege: Hive administrator permission.
    • Hive Read Write Privileges: Hive data table management permission, which is the operation permission to set and manage the data of created tables.
    • Hive role management supports the administrator permission, and the permissions of accessing tables and views, without granting the database permission.
    • The permissions of the Hive administrator do not include the permission to manage HDFS.
    • If there are too many tables in the database or too many files in tables, the permission granting may last a while. For example, if a table contains 10,000 files, the permission granting lasts about 2 minutes.
    Table 3 Configuring a role

    Scenario

    Role Authorization

    Setting the permission to query a table of another user in the default database
    1. In the View Name area, click Hive Read Write Privileges.
    2. Click the name of the specified database in the database list. Tables in the database are displayed.
    3. In the Rights column of the specified table, choose Select.

    Setting the permission to import data to a table of another user in the default database
    1. In the View Name area, click Hive Read Write Privileges.
    2. Click the name of the specified database in the database list. Tables in the database are displayed.
    3. In the Permission column of the specified indexes, select Delete and Insert.

  1. Click OK. Return to the Role page.

    After the role is created, you can create a HetuEngine user and assign related role permissions to the user by referring to Creating a HetuEngine User.

Table 4 describes the permission requirements when SQL statements are processed in HetuEngine.

Table 4 Using HetuEngine tables, columns, or data

Scenario

Required Permission

DESCRIBE TABLE

Select

ANALYZE TABLE

Select and Insert

SHOW COLUMNS

Select

SHOW TABLE STATUS

Select

SHOW TABLE PROPERTIES

Select

SELECT

Select

EXPLAIN

Select

CREATE VIEW

Select, Grant Of Select, and Create

CREATE TABLE

Create

ALTER TABLE ADD PARTITION

Insert

INSERT

Insert

INSERT OVERWRITE

Insert and Delete

ALTER TABLE DROP PARTITION

The table-level Alter and Delete, and column-level Select permissions need to be granted.

ALTER DATABASE

Hive Admin Privilege