Accessing OBS Using an ECS Agency

Updated at: Apr 28, 2020 GMT+08:00

MRS supports IAM's agency mechanism, which enables ECS to automatically obtain temporary AK/SK to access OBS. This prevents the AK/SK from being exposed in the configuration file.

This function is available for components Hadoop, Hive, Spark, HBase, Presto, and Flink in clusters of MRS 2.0.5 or later.

MRS provides two methods for accessing OBS using the obs:// protocol:

Configure the AK/SK in an MRS cluster. The AK/SK will be exposed in the configuration file in plaintext. Exercise caution when performing this operation. For details, see Accessing OBS Using obs.
Bind an agency of the ECS type to an MRS cluster to access OBS, preventing the AK/SK from being exposed in the configuration file. For details, see the following part in this section.

Prerequisites

Enable fine-grained access control in IAM.

(Optional) Step 1: Create an ECS Agency with OBS Access Permissions

MRS presets MRS_ECS_DEFAULT_AGENCY in the agency list of IAM so that you can select this agency when creating a cluster. This agency has the OBS OperateAccess permissions and the CES FullAccess (only available for users who have enabled fine-grained policies), CES Administrator, and KMS Administrator permissions in the region where the cluster resides. Do not modify MRS_ECS_DEFAULT_AGENCY on IAM.
If you want to use the preset agency, skip the step for creating an agency. If you want to use a custom agency, perform the following steps to create an agency. (To create or modify an agency, you must have the Security Administrator permission.)

Log in to the IAM console.
Choose Agencies. On the displayed page, click Create Agency.
Enter an agency name, for example, mrs_ecs_obs.
Set Agency Type to Cloud service and select Elastic Cloud Server (ECS) and Bare Metal Server (BMS) to authorize ECS or BMS to invoke OBS. See Figure 1.
Set Validity Period to Unlimited.
Figure 1 Creating an agency
Click Assign Permissions in the Permissions area.
On the displayed page, search for the OBS OperateAccess policy, select it, and click OK. See Figure 2.
Figure 2 Assigning permissions
Click OK.

Step 2: Bind an ECS Agency to an MRS Cluster

Method 1: Binding an Agency After Creating a Cluster

Log in to the MRS management console. In the left navigation pane, choose Clusters > Active Clusters.
Click the name of the cluster to enter its details page.
On the Dashboard tab page, click on the right side of Agency to select an existing agency. If there is no desired agency, click Create Agency to go to the IAM console to create an agency and select it.
Figure 3 Binding an agency

Method 2: Binding an Agency When Creating a Cluster

Log in to the MRS management console.
Click Buy Cluster. In the upper right corner of the page, click Try the new edition.
Figure 4 Switching to the new edition
Click the Custom Config tab.
Figure 5 Custom purchase of a cluster
Set related parameters. In the Agency area on the Set Advanced Options tab page, select the agency created in (Optional) Step 1: Create an ECS Agency with OBS Access Permissions.
Figure 6 Configuring an agency