On this page

Notes and Constraints
Updating DLI Agency Permissions (dli_management_agency)
Follow-Up Operations
Example 1: Configure Permissions for DLI Usage, Datasource Connection, and O&M Scenarios in Project A
Example 2: Configure Permissions for DLI Usage, Datasource Connection, and O&M Scenarios in Project B

Show all

Help Center/ Data Lake Insight/ User Guide/ Preparations/ Configuring DLI Agency Permissions

Configuring DLI Agency Permissions

Updated on 2024-12-28 GMT+08:00

View PDF

To use DLI, you must first configure permissions.

This section applies to the following scenarios:

If you use DLI for the first time, configure DLI agency permissions by referring to this section.
DLI needs to work with other cloud services. You must grant DLI basic operation permissions of these services so that DLI can access them and perform resource O&M operations on your behalf.
If you are still using the previous-generation agency dli_admin_agency, update it by referring to this section.
To balance practical business needs with the risk of excessive delegation, DLI upgraded its system agency to achieve more granular control over permissions. The previous dli_admin_agency was upgraded to dli_management_agency, which includes permissions for accessing IAM user information, datasource operations, and message notifications. This effectively prevents uncontrolled permission issues related to the services associated with DLI. After the upgrade, the DLI agency is more flexible and more suitable for scenario-based agency customization for medium- and large-sized enterprises.

After agency permissions are configured, the dli_management_agency agency is generated on the Agencies page of the IAM console. Do not delete this default system agency. Otherwise, the permissions included in the agency will be automatically revoked. The system cannot obtain IAM user information, access network resources required by datasource connections, or access SMN to send notifications.

Notes and Constraints

Only the tenant account or a member account of user group admin can authorize the service.
DLI authorization needs to be conducted by project. The permissions of required agencies must be updated separately in each project. This means you need to switch to the corresponding project and then update the agency by following the instructions provided in this section.

Updating DLI Agency Permissions (dli_management_agency)

In the navigation pane of the DLI console, choose Global Configuration > Service Authorization.

On the displayed page, select permissions for scenarios.

Click on a permission card to view its detailed permission policies.

Table 1 describes these agencies.

**Table 1** Permissions contained in the dli_management_agency agency
Use Case	Agency	Description
Basic usage	IAM ReadOnlyAccess	To authorize IAM users who have not logged in to DLI, you need to obtain their information. So, the permissions contained in the IAM ReadOnlyAccess policy are required. IAM ReadOnlyAccess is a global policy. Make sure you select this policy. If you do not select it, all its permissions will become invalid in all regions, and the system cannot obtain IAM user information.
Datasource	DLI Datasource Connections Agency Access	Permissions to access and use VPCs, subnets, routes, and VPC peering connections
O&M	DLI Notification Agency Access	Permissions to send notifications through SMN when a job fails to be executed

NOTE:

Among the permissions contained in dli_management_agency:

The authorization scope of the IAM ReadOnlyAccess policy covers all global service resources in all regions.
- If you select this policy when updating a DLI agency in any region, this policy's permissions apply to the projects in all regions.
- If you do not select this policy when updating an agency in any project, this policy's permissions will be revoked from all regions. This means that all projects cannot obtain IAM user information.
The authorization scope of the DLI Datasource Connections Agency Access and DLI Notification Agency Access policies covers the project resources in specified regions.
These policies' permissions only apply to projects for which these policies are selected and the DLI agency permissions are updated. Projects for which these policies are not selected do not have the permissions required in datasource scenarios and the permission to send notifications using SMN.

Example 1: Configure Permissions for DLI Usage, Datasource Connection, and O&M Scenarios in Project A and Example 2: Configure Permissions for DLI Usage, Datasource Connection, and O&M Scenarios in Project B demonstrate the agency permission differences caused by updating a DLI agency for different projects in a region.

Select the policies to be included in dli_management_agency and click Update.
Figure 1 Updating agency permissions
View and understand the notes for updating the agency and click OK. The DLI agency permissions are updated.
- The system upgrades your dli_admin_agency to dli_management_agency.
- To maintain compatibility with existing job agency permission requirements, dli_admin_agency will still be listed in the IAM agency list even after the update.
- Do not delete the agency created by the system by default.

Follow-Up Operations

In addition to the permissions provided by dli_management_agency, you need to create an agency on the IAM console and add information about the new agency to the job configuration for scenarios like allowing DLI to read and write data from and to OBS to transfer logs, or allowing DLI to access DEW to obtain data access credentials. For details, see Creating a Custom DLI Agency and Agency Permission Policies in Common Scenarios.

When Flink 1.15, Spark 3.3.1 (Spark general queue scenario), or a later version is used to execute jobs, you need to create an agency on the IAM console.
If the engine version is earlier than Flink 1.15, dli_admin_agency is used by default during job execution. If the engine version is earlier than Spark 3.3.1, user authentication information (AK/SK and security token) is used during job execution.
This means that jobs whose engine versions are earlier than Flink 1.15 or Spark 3.3.1 are not affected by the update of agency permissions and do not require custom agencies.

Common service scenarios where you need to create an agency:

Data cleanup agency required for clearing data according to the lifecycle of a table and clearing lakehouse table data. You need to create a DLI agency named dli_data_clean_agency on IAM and grant permissions to it. You need to create an agency and customize permissions for it. However, the agency name is fixed to dli_data_clean_agency.
Tenant Administrator permissions are required to access data from OBS to execute Flink jobs on DLI, for example, obtaining OBS data sources, log dump (including bucket authorization), checkpointing enabling, and job import and export.
The AK/SK required by DLI Flink jobs is stored in DEW. To allow DLI to access DEW data during job execution, you need to create an agency to delegate the permissions to operate on DEW data to DLI.
To allow DLI to access DLI catalogs to retrieve metadata when executing jobs, you need to create a new agency that grants DLI catalog data operation permissions to DLI. This will enable DLI to access DLI catalogs on your behalf.
Cloud data required by DLI Flink jobs is stored in LakeFormation. To allow DLI to access catalogs to retrieve metadata during job execution, you need to create an agency to delegate the permissions to operate on catalog data to DLI.

When creating an agency, you cannot use the default agency names dli_admin_agency, dli_management_agency, or dli_data_clean_agency. It must be unique.

For more information about custom agency operations, see Creating a Custom DLI Agency and Agency Permission Policies in Common Scenarios.

Example 1: Configure Permissions for DLI Usage, Datasource Connection, and O&M Scenarios in Project A

Operation instruction: A DLI user upgrades dli_admin_agency to dli_management_agency for project A in CN North-Beijing4.
1. On the DLI management console, switch to project A in the CN North-Beijing4 region and choose Global Configuration > Service Authorization.
2. Select the policies under Basic Usage, Datasource, and O&M.
  Figure 2 Updating a DLI agency for project A in CN North-Beijing4
3. Click Update.

Permission description: The agency permissions are updated for project A in CN North-Beijing4.

The IAM ReadOnlyAccess policy's permissions are granted to global service resources, meaning that all regions and projects have these permissions.

The DLI Datasource Connections Agency Access and DLI Notification Agency Access policies contain only regional permissions, meaning that their permissions only apply to project A in CN North-Beijing4.

Example of Agency Permissions for Project A in CN North-Beijing4	Example of Agency Permissions for Project B in CN North-Beijing4
dli_management_agency The new agency contains the following policy permissions: IAM ReadOnlyAccess DLI Datasource Connections Agency Access DLI Notification Agency Access	dli_management_agency The new agency contains the following policy permissions: IAM ReadOnlyAccess

Example of Agency Permissions for Project A in CN North-Beijing4

Example of Agency Permissions for Project B in CN North-Beijing4

dli_management_agency

The new agency contains the following policy permissions:

IAM ReadOnlyAccess
DLI Datasource Connections Agency Access
DLI Notification Agency Access

dli_management_agency

The new agency contains the following policy permissions:

IAM ReadOnlyAccess

Example 2: Configure Permissions for DLI Usage, Datasource Connection, and O&M Scenarios in Project B

Operation instruction: To assign the permissions of the DLI Datasource Connections Agency Access and DLI Notification Agency Access policies to project B in the CN North-Beijing4 region, perform the following operations to update the agency permissions of project B in CN North-Beijing4:

On the DLI management console, switch to project B in the CN North-Beijing4 region and choose Global Configuration > Service Authorization.
Select the policies under Basic Usage, Datasource, and O&M.
CAUTION:

When updating the agency permissions for project B, you will need to select the IAM ReadOnlyAccess policy, as its authorization scope covers global service resources. If you deselect it and update the agency, all its permissions will become invalid in all regions and projects.

Figure 3 Updating a DLI agency for project B in CN North-Beijing4
Click Update.

Permission description:

The authorization scope of the DLI Datasource Connections Agency Access and DLI Notification Agency Access policies covers the project resources in specified regions. Following updates to agency permissions, project B in CN North-Beijing4 has the permission to obtain IAM user information, perform datasource operations, and send message notifications.

Example of Agency Permissions in Region A	Example of Agency Permissions in Region B
dli_management_agency The new agency contains the following policy permissions: IAM ReadOnlyAccess DLI Datasource Connections Agency Access DLI Notification Agency Access	dli_management_agency The new agency contains the following policy permissions: IAM ReadOnlyAccess DLI Datasource Connections Agency Access DLI Notification Agency Access

Example of Agency Permissions in Region A

Example of Agency Permissions in Region B

dli_management_agency

The new agency contains the following policy permissions:

IAM ReadOnlyAccess
DLI Datasource Connections Agency Access
DLI Notification Agency Access

dli_management_agency

The new agency contains the following policy permissions:

IAM ReadOnlyAccess
DLI Datasource Connections Agency Access
DLI Notification Agency Access

Parent topic: Preparations

Previous topic: Preparations

Next topic: Creating an IAM User and Granting Permissions

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot