Accessing OBS Using an ECS Agency
MRS supports IAM's agency mechanism, which enables ECS to automatically obtain temporary AK/SK to access OBS. This prevents the AK/SK from being exposed in the configuration file.
This function is available for components Hadoop, Hive, Spark, HBase, Presto, and Flink in clusters of MRS 2.0.5 or later.
MRS provides two methods for accessing OBS using the obs:// protocol:
- Configure the AK/SK in an MRS cluster. The AK/SK will be exposed in the configuration file in plaintext. Exercise caution when performing this operation. For details, see Accessing OBS Using obs.
- Bind an agency of the ECS type to an MRS cluster to access OBS, preventing the AK/SK from being exposed in the configuration file. For details, see the following part in this section.
Enable fine-grained access control in IAM.
(Optional) Step 1: Create an ECS Agency with OBS Access Permissions
- MRS presets MRS_ECS_DEFAULT_AGENCY in the agency list of IAM so that you can select this agency when creating a cluster. This agency has the OBS OperateAccess permissions and the CES FullAccess (only available for users who have enabled fine-grained policies), CES Administrator, and KMS Administrator permissions in the region where the cluster resides. Do not modify MRS_ECS_DEFAULT_AGENCY on IAM.
- If you want to use the preset agency, skip the step for creating an agency. If you want to use a custom agency, perform the following steps to create an agency. (To create or modify an agency, you must have the Security Administrator permission.)
- Log in to the IAM console.
- Choose Agencies. On the displayed page, click Create Agency.
- Enter an agency name, for example, mrs_ecs_obs.
- Set Agency Type to Cloud service and select Elastic Cloud Server (ECS) and Bare Metal Server (BMS) to authorize ECS or BMS to invoke OBS. See Figure 1.
- Set Validity Period to Unlimited.
- Click Assign Permissions in the Permissions area.
- On the displayed page, search for the OBS OperateAccess policy, select it, and click OK. See Figure 2.
- Click OK.
Step 2: Bind an ECS Agency to an MRS Cluster
Method 1: Binding an Agency After Creating a Cluster
- Log in to the MRS management console. In the left navigation pane, choose Clusters > Active Clusters.
- Click the name of the cluster to enter its details page.
- On the Dashboard tab page, click on the right side of Agency to select an existing agency. If there is no desired agency, click Create Agency to go to the IAM console to create an agency and select it.
Figure 3 Binding an agency
Method 2: Binding an Agency When Creating a Cluster
- Log in to the MRS management console.
- Click Buy Cluster. In the upper right corner of the page, click Try the new edition.
Figure 4 Switching to the new edition
- Click the Custom Config tab.
Figure 5 Custom purchase of a cluster
- Set related parameters. In the Agency area on the Set Advanced Options tab page, select the agency created in (Optional) Step 1: Create an ECS Agency with OBS Access Permissions.
Figure 6 Configuring an agency