Managing Pangu Data Assets

Introduction to Data Assets

Data assets refer to datasets that are managed, stored, and available on the platform.

Data assets are classified into the following types:

• User-published datasets

  You can publish a dataset as a data asset by choosing Data Engineering > Data Publishing > Publish Task. You can view details about published datasets, edit and delete datasets, and publish datasets to AI Gallery.

• Data assets subscribed from AI Gallery

  In addition to datasets published by users, the platform also allows you to subscribe to data assets from AI Gallery. AI Gallery enables the sharing of AI assets such as models, datasets, and AI applications. In this community, enterprises, individual developers, and other groups can securely share and purchase AI assets.

Constraints

Before publishing datasets to AI Gallery or subscribing to data assets from AI Gallery, you need to configure OBS authorization on ModelArts. For details about the configuration method, see Configuring OBS Authorization.

Configuring OBS Authorization

1. Log in to the ModelArts service console and choose Permissions in the navigation pane on the left.
2. On the Permissions page, click Add Authorization.
3. On the Add Authorization page, set the parameters by referring to Table 1. The following uses an IAM user as an example to describe how to configure authorization. For details, see Figure 1.

   Table 1 Parameters for adding authorization

   Configuration Item	Configuration Sub-item	Description
   Authorization configuration	Authorized User	Configure the type of the user to be authorized. ModelArts supports IAM users, federated users, agency users, and all users. To obtain OBS authorization of ModelArts, select All users or IAM user.
   	Authorization objects	When Authorized User is set to IAM user, set the IAM user account to be authorized.
   	Agency	Set the agency type. If an agency type has been set in ModelArts, select Existing agency. Otherwise, select Add agency.
   	Agency Name	When you select Add agency, you need to customize the agency name. For details about the agency name format, see the dialog box displayed.
   	Permission configuration	Permission configuration type. Select Common mode.
   Permission template		Preconfigured permission template, which does not need to be selected.
   Service list		Search for OBS in the service list. Object Storage Service (OBS) is displayed. Select it.
   Function permissions		After you select a service from the service list, all permissions of the service are displayed. Select the required permissions.

   Figure 1 Adding OBS Permissions for IAM Users
   
4. To publish data to AI Gallery, you need to obtain the OBS permissions listed in Table 2.

   Table 2 OBS permissions required for publishing data

   Permission	Description
   obs:bucket:ListBucket	Lists objects in a bucket.
   obs:object:GetObject	Obtains object content and metadata.
   obs:bucket:GetBucketLocation	Obtains a bucket's location.

5. To subscribe to data assets from AI Gallery, obtain the OBS permissions listed in Table 3.

   Table 3 OBS permissions required for subscribing to data assets

   Permission	Description
   obs:object:PutObject	Uploads objects with PUT or POST, copies objects, appends data to objects, initiates a multipart upload, as well as uploads and assembles parts.

Publishing Data Assets to AI Gallery

1. Log in to ModelArts Studio Large Model Deveopment Platform. In the My Spaces area, click the required workspace.
   Figure 2 My Spaces
   
2. In the navigation pane, choose Workspace Assets > Data.
3. In the Operation column of the data asset (Publish to Gallery is available), click Publish to Gallery to publish the data asset.
4. In the Publish to AI Gallery dialog box, enter the AI Gallery asset name and description, available subscription area constraint, visible scope, and click OK to publish the data asset to AI Gallery.
   Figure 3 Publishing a Workflow to AI Gallery
   
5. The data asset publishing status is displayed on the data asset list page.
   • If the status is Publishing, the asset is being synchronized to AI Gallery.
   • If the status is Published, the asset has been synchronized to AI Gallery. You can click View Publishing Information in the Operation column to view the publishing information of the asset.
   • If the status is Publish failed, the asset fails to be synchronized to AI Gallery. You can click Publish to AI Gallery to publish the data asset again.

Subscribing to Data Assets from AI Gallery

1. Log in to ModelArts Studio Large Model Deveopment Platform. In the My Spaces area, click the required workspace.
   Figure 4 My Spaces
   
2. In the navigation pane, choose Workspace Assets > Data.
3. Click Subscribe to Data in the upper right corner. In the Subscribe from AI Gallery dialog box, select the data assets to be subscribed to and click Next Step.
4. Enter the asset name and description, and click OK to subscribe to the data asset.
5. The data asset publishing status is displayed on the data asset list page.
   • If the status is Subscribing, the asset is being synchronized from AI Gallery. Wait patiently.
   • If the status is Subscription succeeded, the asset has been successfully subscribed to from AI Gallery. You can click View Subscription Information in the Operation column to view the subscription information of the asset.
   • If the status is Subscription failed, the asset fails to be subscribed to from the AI Gallery. You can click Re-subscribe to subscribe to the data asset from the AI Gallery again.
6. The subscribed data assets are displayed on the Data Engineering > Data Management > Datasets > Original Dataset page. You can process and publish the data assets.

Managing Data Assets

1. Log in to ModelArts Studio Large Model Deveopment Platform. In the My Spaces area, click the required workspace.
   Figure 5 My Spaces
   
2. In the navigation pane, choose Workspace Assets > Data.
3. Click the Task Publishing tab and perform the following operations on the datasets published by users:
   • View basic information. You can click a data asset to view its basic information such as ratio details and data details.
   • Publish data assets to AI Gallery. Click Publish to AI Gallery in the Operation column to publish the data asset to AI Gallery.
   • View the publishing information. Click View Publish Information to view the publishing information of the asset. (This operation can be performed only on the data assets that have been published to AI Gallery.)
   • Editing properties. Choose More > Edit Property in the Operation column to edit the name, description, and visibility of the data asset.
   • Delete a data asset. Choose More > Delete in the Operation column to delete the current data asset.
   • Unpublish data assets from AI Gallery. Choose More > Cancel Publish in the Operation column to unpublish the data asset from AI Gallery. (This operation can be performed only on the data assets that have been published to AI Gallery.)
4. Click the AI Gallery tab and perform the following operations on the data assets subscribed from AI Gallery:
   • View the subscription information. Click View Subscription Information in the Operation column of a specific data asset to view the subscription information such as the asset name and description.
   • Edit properties. Choose More > Edit Properties in the Operation column to edit the name, description, and visibility of the data asset.
   • Delete a data asset. Choose More > Delete in the Operation column to delete the current data asset.
   • Re-subscribe to a data asset. If the subscription fails, click Resubscribe in the Operation column to subscribe to the required data asset from AI Gallery again.