Configuring Queue Permissions

Prerequisites

• Only the DAYU Administrator, Tenant Administrator, or data security administrator has the permission to allocate available queues to the current workspace, configure MRS queue attributes (offline/real-time), and configure user permission policies for specified queues. The workspace administrator can configure queue permission policies for user groups and users.
• Before configuring queue permissions, you have created an MRS Ranger and a DLI connection in Management Center. For details, see Creating a Data Connection.
• Before configuring permissions for MRS Yarn queues, you have synchronized user information from IAM to the data source based on Synchronizing Users.
• To make the permission policy for MRS Yarn queues take effect, you have enabled Yarn access control by setting the yarn.acl.enable parameter to true. For details, see Reference: Configuring Strict Permission Control for Yarn.

Constraints

• Currently, only MRS Yarn queues can be allocated. Permission management is supported only for MRS Yarn and DLI queues. Authorization for the DLI default queue is not supported due to DLI limitations.
• Permissions of MRS Yarn queues can be managed only when the version of the CDM cluster selected as the agent for the data connection is 2.10.0.300 or later.
• Only the DAYU Administrator, Tenant Administrator, or data security administrator has the permission to allocate available queues to the current workspace, configure MRS queue attributes (offline/real-time), and configure user permission policies for specified queues. The workspace administrator can configure queue permission policies for user groups and users.
• The queues allocated to the current workspace are not associated with the configured queue permissions policies which are contained in the data source configuration. Therefore, if the queues are deleted from the current workspace, the configured queue permission policies still take effect. When the queues are added again, the permissions are still available.
• The configured queue permission policies are implemented based on the permission control capability of the data source. You can view the configured policies in the data source (such as MRS Ranger policies and DLI queue management). If you delete a queue policy from the data source, the policy will not be automatically deleted from the DataArts Security component. You need to manually delete the policy from the DataArts Security component.
• Queue attributes (offline or real-time) can be configured only for MRS Yarn queues, and different attributes can be configured for the same queue in different workspaces.
• Due to DLI limitations, permissions of DLI queues can be granted only to users, but not to user groups.

Allocating Queues and Granting Permissions

1. On the DataArts Studio console, locate an instance and click Access. On the displayed page, locate a workspace and click DataArts Security.
   Figure 1 DataArts Security
   

2. In the left navigation pane, choose Queue Permissions.
   Figure 2 Queue Permissions page
   

3. Click above the queue permission directory to allocate a queue to the current workspace. In the displayed Add Queue Resource dialog box, set the parameters listed in Table 1 and click Save.

   Table 1 Parameters for adding a queue

   Parameter	Description
   *Resource Type	Select MRS queues or DLI queues.
   *Data Connection	Select the data connection where the queue is located. For details about how to create a data connection, see Creating a Data Connection.
   *Cluster Name	This parameter is displayed only when Resource Type is set to MRS queues. The system automatically matches the cluster name corresponding to the data connection.
   *Queue Name	Select the queue to be authorized. If you set Resource Type to MRS queues, the available queues are from an MRS cluster. To view the available queues, go to the MRS console, click a cluster name to go to the cluster details page, and click the Tenants and then Queue Configuration tab. If you set Resource Type to DLI queues, the available queues are the queues purchased in DLI. To view the available queues, go to the DLI console and choose Resources > Queue Management. In addition, DLI queues are classified into SQL queues and general-purpose queues. SQL queues are used to run SQL jobs, and general-purpose queues are used to run Flink and Spark JAR jobs.
   Description	Information to make the queue easier to be identified

   Figure 3 Adding queues
   

4. Click a queue in the queue permission directory to go to the queue details page.
   You can configure attributes for MRS Yarn queues, which are mainly used for task management in DataArts Factory. Real-time queues are used to run real-time jobs, and offline queues are used to run batch jobs. By default, job types of queues are not distinguished.

   Figure 4 MRS Yarn queue details
   
   Figure 5 DLI queue details
   

5. Grant permissions to the allocated queues.
   • MRS Yarn queue
     On the MRS Yarn queue details page, click Create Policy. In the displayed dialog box, set the parameters in Table 2 and click Save.

     Table 2 MRS Yarn queue policy parameters

     Parameter	Description
     Cluster Name	The system automatically sets this parameter to the name of the cluster where the queue is located.
     Queue Name	The system automatically sets this parameter to the current queue name.
     *Policy Name	Name of the permission policy for the MRS Yarn queue. To facilitate policy management, you are advised to include the authorization object in the name.
     Policy Description	Information to make the policy easier to be identified
     Policy Status	If this function is enabled, the current policy takes effect.
     Audit Log	If this function is enabled, operation logs of the current queue can be recorded. You can view the audit logs in the data source.
     Overwrite	Due to the restrictions of the Ranger component, if a queue permission policy already exists for the user or user group in the Ranger component, the current policy may be considered duplicate and cannot be added. If this function is enabled, the system attempts to overwrite the existing queue permission policy for the user or user group in Ranger. If the overwriting fails, you need to delete the queue permission policy of the user or user group from the Ranger component and add the policy again.
     *Access Authorization (Click Add User to open the configuration window.)
     Username	Select the users or user groups to be authorized. The users and user groups that have been added to the workspace are available for selection.
     Permission	submit-app: the permission required for submitting queues admin-queue: the permission required for managing queues
     Agency	If you want the users or user groups to be authorized to manage this policy, you can enable this option so that the users or user groups become the administrators of this policy and can update or delete the policy.

     Figure 6 MRS Yarn queue details
     

   • DLI queue

     On the DLI queue details page, click Authorize. In the displayed dialog box, set the parameters in Table 2 and click Save.

     Table 3 DLI queue authorization parameters

     Parameter	Description
     Username	Select the users to be authorized. The users that have been added to the workspace are available for selection. NOTE: Permissions of DLI queues can be granted only to users, but not to user groups.
     Permissions	Submitting jobs: This permission allows you to submit jobs to this queue. Terminating jobs: This permission allows you to terminate jobs submitted to this queue. Deleting queues: This permission allows you to delete the queue. Granting permissions: This permission allows you to grant queue permissions to other users. Revoking permissions: This permission allows you to revoke the queue permissions from other users except the queue owner. Viewing other users' permissions: This permission allows you to view the queue permissions of other users. Restarting queues: This permission allows you to restart the queue. Modifying queue specifications: This permission allows you to modify queue specifications.

     Figure 7 DLI queue details
     

Related Operations

• Deleting queues: In the queue permission directory, select queues and click to delete them.
  

  • When a queue is deleted, it is not directly deleted from MRS or DLI. Instead, the queue will no longer be allocated to the workspace.
  • After a queue is deleted, the permissions configured for the queue are still valid. For how to delete queue permissions, see Deleting policies or Revoking permissions.
  • Yarn queues that are being used in DataArts Factory cannot be deleted in DataArts Security.
• Editing policies: On the MRS Yarn queue details page, locate a policy and click Edit in the Operation column to edit the policy.
• Deleting policies: On the MRS Yarn queue details page, locate a policy and click Delete in the Operation column to delete the policy. To delete multiple policies at a time, select the policies and click Delete above the policy list.
  

  The deletion operation cannot be undone. Exercise caution when performing this operation.

• Modifying permissions: On the DLI queue details page, locate a permission and click Modify in the Operation column.
• Revoking permissions: On the DLI queue details page, locate a permission and click Revoke in the Operation column.

Reference: Configuring Strict Permission Control for Yarn

• The procedure is as follows:
  1. Log in to FusionInsight Manager and choose Cluster > Services > Yarn.
  2. On the displayed page, click the Configuration tab then the All Configurations sub-tab. On this sub-tab page, search for the yarn.acl.enable parameter and change its value to true. If the value is true, no further action is required.
     Figure 8 Configuring yarn.acl.enable
     

Before configuring permissions for Yarn queues, you need to enable permission control for Yarn queues.

1. Log in to MRS FusionInsight Manager.
2. Choose Cluster > Services > Yarn and click Configurations and then Basic Configurations. Search for the yarn.acl.enable parameter and change its value to true. If the value is true, no further action is required.
   Figure 9 Configuring the yarn.acl.enable parameter
   

3. After the parameter is set, click Save in the upper left corner and then OK in the dialog box to save the configuration.
4. After saving the configuration, switch to the Instances tab page, select the instance that has expired, click More, and select Instance Rolling Restart to make the configuration take effect.
   Figure 10 Performing a rolling instance restart