HetuEngine User Permissions

Permission Principles and Constraints

• Accessing data sources in the same cluster using HetuEngine

  If Ranger authentication is enabled for HetuEngine, the PBAC permission policy of Ranger is used for authentication.

  If Ranger authentication is disabled for HetuEngine, the RBAC permission policy of MetaStore is used for authentication.

• Accessing data sources in different clusters using HetuEngine

  The permission policy is controlled by the permissions of the HetuEngine client and the data source. (In Hive scenarios, it depends on HDFS.)

• When querying a view, you only need to grant the select permission on the target view. When querying a join table using a view, you need to grant the select permission on the view and table.
• Columns in GaussDB and HetuEngine data sources cannot be masked.

When the permission control type of HetuEngine is changed, the HetuEngine service, including the HetuEngine compute instance running on the HSConsole page, needs to be restarted.

HetuEngine Ranger-based Permission Control

By default, Ranger authentication is used for newly installed clusters. For clusters upgraded from earlier versions or clusters where Ranger authentication is manually disabled, you can enable Ranger authentication again by referring to the following content.

For a cluster with Ranger authentication enabled, cluster administrators can use Ranger to configure the permissions to manage databases, tables, and columns of data sources for HetuEngine users. For details, see Adding a Ranger Access Permission Policy for HetuEngine.

1. Log in to FusionInsight Manager.
2. If Kerberos authentication is disabled for the cluster (the cluster is in normal mode), add the ranger.usersync.sync.source parameter. If Kerberos authentication is enabled for the cluster (the cluster is in security mode), skip this step.
   1. Choose Cluster > Services > Ranger. Click Configurations then All Configurations.
   2. Search for the ranger.usersync.config.expandor parameter, set its name to ranger.usersync.sync.source, set its value to ldap, and save the settings.
   3. On the Dashboard page, click More > Restart Service in the upper right corner, enter the password, and restart Ranger.
      

      For MRS 3.5.0 and later versions, run the following commands:

      To use Ranger for permission control when Kerberos authentication is disabled for the cluster (the cluster is in normal mode), choose Cluster > Services > Ranger > Configurations > All Configurations, search for ranger.usersync.sync.source, and ensure that the value is ldap, otherwise, change the value to ldap. Save the modification and restart Ranger.

3. Choose Cluster > Services > HetuEngine > More > Enable Ranger.
4. Choose Cluster > Services > HetuEngine. Click More and select Restart Service.
5. Restart the compute instance on HSConsole.

HetuEngine MetaStore-based Permission Control

• Constraints: This function applies only to Hive data sources.

  When multiple HetuEngine clusters are deployed for collaborative computing, the metadata is centrally managed by the management cluster. Data computing is performed in all clusters. The user permission for accessing HetuEngine clusters must be configured in the management cluster. Users who belong to the Hive user group and share the same name are added to all compute instances.

• Enabling MetaStore Authentication
  1. Log in to FusionInsight Manager.
  2. Choose Cluster > Services > HetuEngine. Click More and select Disable Ranger.
  3. Choose Cluster > Services > HetuEngine. Click More and select Restart Service.
  4. Restart the compute instance on the HSConsole page.
• MetaStore Permission

  Similar to Hive, HetuEngine is a data warehouse framework built on Hadoop, providing storage of structured data like SQL.

  Permissions in a cluster must be assigned to roles which are associated to users or user groups. Users can obtain permissions only by binding a role or joining a group that is bound with a role.

  HetuEngine permission management is performed by the permission system to manage users' operations on the database, ensuring that different users can operate databases independently and securely. A user can operate another user's tables and databases only with the corresponding permissions. Otherwise, operations will be rejected.

  HetuEngine permission management integrates the functions of Hive permission management. MetaStore service of Hive and the function of granting permissions on the web page are required to enable the HetuEngine permission management.

  • Granting permissions on the web page: HetuEngine supports only granting permissions on the web page. On Manager, choose System > Permission to add or delete a user, user group, or a role, and to grant or cancel permissions.
  • Obtaining and judging a service: When the DDL and DML commands are received from the client, HetuEngine will obtain the client user's permissions on database information from MetaStore, and check whether the required permissions are included. If the required permissions have been obtained, the user's operations are allowed. If the permissions are not obtained, the user's operation will be rejected. After the MetaStore permission check is passed, ACL permission also needs to be checked on HDFS.
• HetuEngine Permission Model

  If a user uses HetuEngine to perform SQL query, the user must be granted with permissions of HetuEngine databases and tables (include external tables and views). The complete permission model of HetuEngine consists of the metadata permission and HDFS file permission. Permissions required to use a database or a table are just one type of HetuEngine permission.

  • Metadata permissions

    Metadata permissions are controlled at the metadata level. Similar to traditional relational databases, the HetuEngine database contains the CREATE and SELECT permissions. Tables and columns contain the SELECT, INSERT, UPDATE, and DELETE permissions. HetuEngine also supports the owner permission OWNERSHIP and cluster administrator permission ADMIN.

  • Data file permissions (that is, HDFS file permissions)

    HetuEngine database and table files are stored in HDFS. The created databases or tables are saved in the /user/hive/warehouse directory of HDFS by default. The system automatically creates subdirectories named after database names and database table names. To access a database or a table, the corresponding file permissions (read, write, and execute) on the HDFS are required.

  To perform various operations on HetuEngine databases or tables, you need to associate the metadata permission and the HDFS file permission. For example, to query HetuEngine data tables, you need to associate the metadata permission SELECT with the READ and EXECUTE permissions on HDFS files.

  To use the management function of FusionInsight Manager GUI to manage the permissions of HetuEngine databases and tables, you only need to configure the metadata permission, and the system will automatically associate and configure the HDFS file permission. In this way, operations on the interface are simplified, improving efficiency.

• HetuEngine Application Scenarios and Related Permissions

  A user needs to join in the Hive group if a database is created using the HetuEngine service, and role authorization is not required. Users have all permissions on the databases or tables created by themselves in Hive or HDFS. They can create tables, select, delete, insert, or update data, and grant permissions to other users to allow them to access the tables and corresponding HDFS directories and files.

  A user can access the tables or database only with permissions. Permissions required for the user vary depending on different HetuEngine scenarios.

  Table 2 Typical HetuEngine scenarios and required permissions

  Typical Scenario	Required Permission
  Using HetuEngine tables, columns, or databases	Permissions required in different scenarios are as follows: To create a table, the CREATE permission is required. To query data, the SELECT permission is required. To insert data, the INSERT permission is required.

  In some special HetuEngine scenarios, other permissions must be configured separately.

  Table 3 Typical HetuEngine authentication scenarios and required permissions

  Scenario	Required Permission
  Creating HetuEngine databases, tables, and foreign tables, or adding partitions to created tables or foreign tables when data files specified by Hive users are saved to other HDFS directories except /user/hive/warehouse.	The directory must exist, the client user must be the owner of the directory, and the user must have the Read, Write, and Execute permissions on the directory. The user must have the Read and Execute permissions of all the upper-layer directories of the directory.
  Performing operations on all databases and tables in Hive	The user must be added to the supergroup user group, and be assigned the ADMIN permission.

• Configuring Permissions for SparkSQL Tables, Columns, and Databases

  After MetaStore authentication is enabled, if a user needs to access HetuEngine tables or databases created by other users, the user needs to be granted with related permissions. HetuEngine supports permission control based on columns for strict permission control. If a user needs to access some columns in tables created by other users, the user must be granted the permission for columns.

  

  • Any permission for a table in the database is automatically associated with the HDFS permission for the database directory to facilitate permission management. When any permission for a table is canceled, the system does not automatically cancel the HDFS permission for the database directory to ensure performance. In this case, users can only log in to the database and view table names.
  • When the query permission on a database is added to or deleted from a role, the query permission on tables in the database is automatically added to or deleted from the role. This mechanism is inherited from Hive.
  • In HetuEngine, the name of a column of the struct type data cannot contain special characters, that is, characters other than letters, digits, and underscores (_). If the column name of the struct data type contains special characters, the column cannot be displayed on the FusionInsight Manager console when you grant permissions to roles on the Role page.

  Procedure

  1. Log in to FusionInsight Manager.
  2. Choose System > Permission > Role.
  3. Click Create Role, and set Role Name and Description.
  4. In the Configure Resource Permission area, choose Name of the desired cluster > Hive and set role permissions. For details, see Table 4.
     • Hive Admin Privilege: Hive administrator permission.
     • Hive Read Write Privileges: Hive data table management permission, which is the operation permission to set and manage the data of created tables.
     

     • Hive role management supports the administrator permission, and the permissions of accessing tables and views, without granting the database permission.
     • The permissions of the Hive administrator do not include the permission to manage HDFS.
     • If there are too many tables in the database or too many files in tables, the permission granting may last a while. For example, if a table contains 10,000 files, the permission granting lasts about 2 minutes.

     Table 4 Setting a role

     Task	Operation
     Setting the permission to query a table of another user in the default database	In the View Name area, click Hive Read Write Privileges. Click the name of the specified database in the database list. Tables in the database are displayed. In the Permission column of a specified table, choose Select.
     Setting the permission to import data to a table of another user in the default database	In the View Name area, click Hive Read Write Privileges. Click the name of the specified database in the database list. Tables in the database are displayed. In the Permission column of the specified indexes, select Delete and Insert.

  5. Click OK. Return to the Role page.
     

     After the role is created, you can create a HetuEngine user and assign related role permissions to the user by referring to Creating a HetuEngine Permission Role.

  Table 5 describes the permission requirements when SQL statements are processed in HetuEngine.

  Table 5 Using HetuEngine tables, columns, or data

  Scenario	Required Permission
  DESCRIBE TABLE	Select
  ANALYZE TABLE	Select and Insert
  SHOW COLUMNS	Select
  SHOW TABLE STATUS	Select
  SHOW TABLE PROPERTIES	Select
  SELECT	Select
  EXPLAIN	Select
  CREATE VIEW	Select, Grant Of Select, and Create
  CREATE TABLE	Create
  ALTER TABLE ADD PARTITION	Insert
  INSERT	Insert
  INSERT OVERWRITE	Insert and Delete
  ALTER TABLE DROP PARTITION	Table-level Alter and Delete, and column-level Select
  ALTER DATABASE	Hive Admin Privilege