Preparations

Purchasing Service Resources

Assigning Permissions

1. Configure IAM permissions.
   1. Use a Huawei Cloud tenant account to create a developer user group user_group and add developer accounts to user_group. For details, see Step 1 Create a User Group and Add Users to the User Group.
   2. Create a custom policy.
      1. Log in to the management console using a Huawei Cloud tenant account, hover over your username in the upper right corner, and click Identity and Access Management from the drop-down list to switch to the IAM management console.
      2. In the navigation pane on the left, choose Permissions > Policies/Roles. Click Create Custom Policy in the upper right corner. On the displayed page, enter Policy1 or Policy2 for Policy Name, select JSON for Policy View, configure the policy content, and click OK.
         

         To define permissions required to access both global and project-level services, enclose the permissions in two separate custom policies for refined authorization. Learn more.

         • The content of Policy1 is as follows:

           {
               "Version": "1.1",
               "Statement": [
                   {
                       "Action": [
                           "modelarts:*:*"
                       ],
                       "Effect": "Allow"
                   },
                   {
                       "Action": [
                           "modelarts:pool:create",
                           "modelarts:pool:update",
                           "modelarts:pool:delete"
                       ],
                       "Effect": "Deny"
                   },
                   {
                       "Action": [
                           "sfsturbo:*:*",
                           "vpc:*:*",
                           "dss:*:get",
                           "dss:*:list"
                       ],
                       "Effect": "Allow"
                   },
                   {
                       "Action": [
                           "ecs:*:*",
                           "evs:*:get",
                           "evs:*:list",
                           "evs:volumes:create",
                           "evs:volumes:delete",
                           "evs:volumes:attach",
                           "evs:volumes:detach",
                           "evs:volumes:manage",
                           "evs:volumes:update",
                           "evs:volumes:use",
                           "evs:volumes:uploadImage",
                           "evs:snapshots:create",
                           "vpc:*:get",
                           "vpc:*:list",
                           "vpc:networks:create",
                           "vpc:networks:update",
                           "vpc:subnets:update",
                           "vpc:subnets:create",
                           "vpc:ports:*",
                           "vpc:routers:get",
                           "vpc:routers:update",
                           "vpc:securityGroups:*",
                           "vpc:securityGroupRules:*",
                           "vpc:floatingIps:*",
                           "vpc:publicIps:*",
                           "ims:images:create",
                           "ims:images:delete",
                           "ims:images:get",
                           "ims:images:list",
                           "ims:images:update",
                           "ims:images:upload"
                       ],
                       "Effect": "Allow"
                   },
                   {
                       "Action": [
                           "vpc:*:*",
                           "ecs:*:get*",
                           "ecs:*:list*"
                       ],
                       "Effect": "Allow"
                   },
                   {
                       "Action": [
                           "kms:cmk:*",
                           "kms:dek:*",
                           "kms:grant:*",
                           "kms:cmkTag:*",
                           "kms:partition:*"
                       ],
                       "Effect": "Allow"
                   }
               ]
           }

         • The content of Policy2 is as follows:

           {
               "Version": "1.1",
               "Statement": [
                   {
                       "Action": [
                           "obs:bucket:ListAllMybuckets",
                           "obs:bucket:HeadBucket",
                           "obs:bucket:ListBucket",
                           "obs:bucket:GetBucketLocation",
                           "obs:object:GetObject",
                           "obs:object:GetObjectVersion",
                           "obs:object:PutObject",
                           "obs:object:DeleteObject",
                           "obs:object:DeleteObjectVersion",
                           "obs:object:ListMultipartUploadParts",
                           "obs:object:AbortMultipartUpload",
                           "obs:object:GetObjectAcl",
                           "obs:object:GetObjectVersionAcl"
                       ],
                       "Effect": "Allow"
                   }
               ]
           }

   3. Grant the custom policy to the developer user group user_group.
      1. In the navigation pane of the IAM console, choose User Groups. On the User Groups page, locate the row that contains user_group, click Authorize in the Operation column, and select Policy1, Policy2, and SWR Admin. Click Next.
         

         SoftWare Repository for Container (SWR) permissions include SWR FullAccess, SWR OperateAccess, and SWR ReadOnlyAccess. However, these permissions are available only for SWR Enterprise Edition, which has been suspended OBT. You need to select SWR Admin.

      2. Set the minimum authorization scope, select All resources for Scope and click OK.

   For details about permission management, see Basic Concepts.

2. Configure ModelArts agency permissions.
   Configure ModelArts agency permissions to allow ModelArts to access dependent services such as OBS.

   1. Log in to the ModelArts console using your Huawei Cloud account. In the navigation pane on the left, choose Settings. On the Global Configuration page, click Add Authorization.
   2. Configure the parameters as follows on the displayed page:

      Authorized User: All users.

      Agency: Add agency.

      Permissions: Common User.

      Select "I have read and agree to the ModelArts Service Statement" and click Create.

      Figure 1 Configuring an agency
      
   3. After the configuration, view the agency configurations of your account on the Global Configuration page.
      Figure 2 Viewing agency configurations
      

3. Configure SWR organization permissions.

   Grant permissions to IAM users in an organization so that they can read, edit, and manage all images in the organization.

   Only accounts and IAM users who have the Manage permission can add permissions for other users.

   1. Log in to the SWR console.
   2. In the navigation pane on the left, choose Organization Management. On the displayed page, click the target organization in the list.
   3. In the Users tab, click Add Permission. In the displayed dialog box, enter an IAM username, select permissions for the user and click OK.

   For details about SWR authorization management, see Granting Permissions of an Organization.

   

   You need to grant the SWR organization permission to the IAM user if they do not have the SWR Admin permission.

4. Verify user permissions.

   You may need to wait for 30 minutes before the permission settings are applied. Then, verify the configuration.

   1. Log in to the ModelArts management console as an IAM in UserGroup-2. On the login page, ensure that IAM User Login is selected.

      Change the password as prompted upon the first login.

   2. Verify ModelArts permissions.
      1. In the upper left corner of the service list, select ModelArts. The ModelArts management console is displayed.
      2. On the ModelArts management console, check whether you can create notebook instances and training jobs, and register images.
   3. Verify SFS permissions.
      1. In the upper left corner of the service list, select SFS. The SFS management console is displayed.
      2. Click Create File System in the upper right corner. If this operation is successful, you have obtained OBS operation permissions.
   4. Verify ECS permissions.
      1. In the upper left corner of the service list, select ECS. The ECS management console is displayed.
      2. Click Buy ECS in the upper right corner. If this operation is successful, you have obtained ECS operation permissions.
   5. Verify VPC permissions.
      1. In the upper left corner of the service list, select VPC. The VPC management console is displayed.
      2. Click Create VPC in the upper right corner. If this operation is successful, you have obtained VPC operation permissions.
   6. Verify DEW permissions.
      1. In the upper left corner of the service list, select DEW. The DEW management console is displayed.
      2. Choose Key Pair Service > Private Key Pairs and click Create Key Pair. If this operation is successful, you have obtained DEW operation permissions.
   7. Verify OBS permissions.
      1. In the upper left corner of the service list, select OBS. The OBS management console is displayed.
      2. Click Create Bucket in the upper right corner. If this operation is successful, you have obtained OBS operation permissions.
   8. Verify SWR permissions.
      1. In the upper left corner of the service list, select SWR. The SWR management console is displayed.
      2. If an SWR page can be properly displayed, you have obtained SWR operation permissions.
      3. Click upload an image in the upper right corner. If authorized organizations are displayed, you have obtained the SWR organization permission.

Creating a Dedicated Resource Pool

ModelArts provides dedicated compute resources, which can be used for notebook instances, training jobs, and model deployment. The resources provided in a dedicated resource pool are exclusive, featuring higher resource efficiency than a public resource pool. To use a dedicated resource pool, create one. For details, see Creating a Standard Dedicated Resource Pool.

• Network: Select the network that has been connected to the VPC. To create a network and interconnect with VPC, see Configuring the Standard Dedicated Resource Pool to Access the Internet.
• Specifications Type and Nodes: Configure the values based on your needs.

Mounting an SFS Turbo File System to an ECS

After you mount an SFS Turbo file system to an ECS, you can upload the training data to SFS Turbo through the ECS. To do so, perform the following operations:

1. Check the cloud service environment.
   • The ECS and the shared SFS disk belong to the same VPC or interconnected VPCs.
   • The base image of the ECS server is Ubuntu 18.04.
   • The ECS and SFS Turbo are in the same subnet.
2. Set the Huawei Cloud image source in the ECS.

   sudo sed -i "s@http://.*archive.ubuntu.com@http://repo.huaweicloud.com@g" /etc/apt/sources.list
   sudo sed -i "s@http://.*security.ubuntu.com@http://repo.huaweicloud.com@g" /etc/apt/sources.list

3. Install the NFS client and mount the target disk.

   sudo apt-get update
   sudo apt-get install nfs-common

4. Obtain the command for mounting the SFS Turbo file system.
   1. Log in to the SFS console.
   2. In the navigation pane on the left, choose SFS Turbo > File Systems. Then, click the file system name to view its details.
   3. In the Basic Info tab, obtain and copy the Linux mounting command.
5. Mount the NFS storage to the ECS server.

   Ensure that the corresponding directory exists and run the following commands:

   mkdir -p /mnt/sfs_turbo
   mount -t nfs -o vers=3,nolock 192.168.0.169:/ /mnt/sfs_turbo

Granting the Read Permission to ModelArts Users on the ECS

When a custom image is used for training on ModelArts, the default user is ma-user and the default user group is ma-group. If a file in the ECS is called during training, grant the read permission to ma-user. Otherwise, error "Permission denied" will be displayed.

1. Create ma-user and ma-group in the ECS.

   default_user=$(getent passwd 1000 | awk -F ':' '{print $1}') || echo "uid: 1000 does not exist" && \
   default_group=$(getent group 100 | awk -F ':' '{print $1}') || echo "gid: 100 does not exist" && \
   if [ ! -z ${default_group} ] && [ ${default_group} != "ma-group" ]; then \
       groupdel -f ${default_group}; \
       groupadd -g 100 ma-group; \
   fi && \
   if [ -z ${default_group} ]; then \
       groupadd -g 100 ma-group; \
   fi && \
   if [ ! -z ${default_user} ] && [ ${default_user} != "ma-user" ]; then \
       userdel -r ${default_user}; \
       useradd -d /home/ma-user -m -u 1000 -g 100 -s /bin/bash ma-user; \
       chmod -R 750 /home/ma-user; \
   fi && \
   if [ -z ${default_user} ]; then \
       useradd -d /home/ma-user -m -u 1000 -g 100 -s /bin/bash ma-user; \
       chmod -R 750 /home/ma-user; \
   fi && \
   # set bash as default
   rm /bin/sh && ln -s /bin/bash /bin/sh

2. Check the created user information.

   id ma-user

   If the following information is displayed, the creation is successful:

   uid=1000(ma-user) gid=100(ma-group) groups=100(ma-group)

Installing and Configuring the OBS CLI

obsutil is a command line tool for accessing and managing OBS. You can use this tool to perform common configuration and management operations on OBS, such as creating buckets, as well as uploading, downloading, and deleting files/folders.

For details about how to install and configure obsutil, see obsutil Quick Start.

Replace the AK/SK and endpoint in the commands with the actual values.

(Optional) Configuring Workspaces

ModelArts allows you to set fine-grained permissions for IAM users and resource isolation between different workspaces. ModelArts workspaces support project resource isolation and settlement of different projects.

If you have enabled the enterprise project function, you can bind an enterprise project ID when creating a workspace, add a user group to the enterprise project, and set fine-grained permissions for users in the group.

If you have not enabled the enterprise project function, create an independent workspace on ModelArts. The enterprise project functions are unavailable.

The workspace is a whitelist function. To use this function, submit a service ticket.