Updated on 2024-12-11 GMT+08:00

Configuring User Permissions for Spark Tables, Columns, and Databases

Scenario

You can configure related permissions if you need to access tables or databases created by other users. SparkSQL supports column-based permission control. If a user needs to access some columns in tables created by other users, the user must be granted the permission for columns. The following describes how to grant table, column, and database permissions to users by using the role management function of Manager.

Procedure

The operations for granting permissions on SparkSQL tables, columns, and databases are the same as those for Hive. For details, see Hive User Permission Management.

  • Any permission for a table in the database is automatically associated with the HDFS permission for the database directory to facilitate permission management. When any permission for a table is canceled, the system does not automatically cancel the HDFS permission for the database directory to ensure performance. In this case, users can only log in to the database and view table names.
  • When the query permission on a database is added to or deleted from a role, the query permission on tables in the database is automatically added to or deleted from the role. This mechanism is inherited from Hive.
  • In Spark, the column name of the struct data type cannot contain special characters, that is, characters other than letters, digits, and underscores (_). If the column name of the struct data type contains special characters, the column cannot be displayed on the FusionInsight Manager console when you grant permissions to roles on the role page.

Concepts

SparkSQL statements are processed in SparkSQL. Table 1 describes the permission requirements.

Table 1 Scenarios of using SparkSQL tables, columns, or databases

Scenario

Required Permission

CREATE TABLE

CREATE, RWX+ownership (for creating external tables - the location)

NOTE:

When creating datasource tables in a specified file path, the RWX and ownership permission on the file next to the path is required.

DROP TABLE

Ownership (of table)

DROP TABLE PROPERTIES

Ownership

DESCRIBE TABLE

Select

SHOW PARTITIONS

Select

ALTER TABLE LOCATION

Ownership, RWX+ownership (for new location)

ALTER PARTITION LOCATION

Ownership, RWX+ownership (for new partition location)

ALTER TABLE ADD PARTITION

Insert, RWX and ownership (for partition location)

ALTER TABLE DROP PARTITION

Delete

ALTER TABLE(all of them except the ones above)

Update and Ownership

TRUNCATE TABLE

Ownership

CREATE VIEW

Select, Grant Of Select, and CREATE

ALTER VIEW PROPERTIES

Ownership

ALTER VIEW RENAME

Ownership

ALTER VIEW ADD PARTS

Ownership

ALTER VIEW AS

Ownership

ALTER VIEW DROPPARTS

Ownership

ANALYZE TABLE

Select and Insert

SHOW COLUMNS

Select

SHOW TABLE PROPERTIES

Select

CREATE TABLE AS SELECT

Select and CREATE

SELECT

Select

NOTE:

The same as tables, you need to have the Select permission on a view when performing a SELECT operation on the view.

INSERT

Insert and Delete (for overwrite)

LOAD

Insert, Delete, and RWX+ownership(input location)

SHOW CREATE TABLE

Select and Grant Of Select

CREATE FUNCTION

ADMIN

DROP FUNCTION

ADMIN

DESC FUNCTION

-

SHOW FUNCTIONS

-

MSCK (metastore check)

Ownership

ALTER DATABASE

ADMIN

CREATE DATABASE

-

SHOW DATABASES

-

EXPLAIN

Select

DROP DATABASE

Ownership

DESC DATABASE

-

CACHE TABLE

Select

UNCACHE TABLE

Select

CLEAR CACHE TABLE

ADMIN

REFRESH TABLE

Select

ADD FILE

ADMIN

ADD JAR

ADMIN

HEALTHCHECK

-