Updated on 2024-06-18 GMT+08:00

Listing Parallel File Systems (SDK for Java)

Function

PFS, a sub-product of OBS, is a high-performance file system with only milliseconds of latency. PFS supports TB/s bandwidth and millions of IOPS, which make it ideal for processing high-performance computing (HPC) workloads.

You can access data in a PFS using standard OBS APIs. You can also read and write data using obsfs, a PFS client that supports POSIX. obsfs can be deployed on a Linux ECS. Then, you can use obsfs to mount a PFS to the ECS and perform a series of operations online, including creating, deleting, and renaming files or directories, and modifying files.

This API returns a list of existing PFSs.

If you have any questions during development, post them on the Issues page of GitHub.

Restrictions

  • To list PFSs, you must have the obs:bucket:ListAllMyBuckets permission. IAM is recommended for granting permissions. For details, see IAM Custom Policies.
  • The mapping between OBS regions and endpoints must comply with what is listed in Regions and Endpoints.
  • PFSs do not support quota configuration. By default, there is no quota limit.
  • To list PFSs, the bucketType parameter must be set to PFS.
  • To query the PFS location when listing PFSs, you need to set ListBucketsRequest.setQueryLocation to true.

Method

obsClient.listBuckets(ListBucketsRequest request)

Request Parameters

Table 1 List of request parameters

Parameter

Type

Mandatory (Yes/No)

Description

request

ListBucketsRequest

Yes

Explanation:

Request parameters for obtaining the PFS list. For details, see Table 2.

Table 2 ListBucketsRequest

Parameter

Type

Mandatory (Yes/No)

Description

queryLocation

boolean

No

Explanation:

Whether to query the PFS location.

Value range:

  • true: The PFS location is queried.
  • false: The PFS location is not queried.

Default value:

false

bucketType

BucketTypeEnum

No

Explanation:

Type of the buckets to be listed.

Value range:

PFS

Default value:

None

Responses

Table 3 ListBucketsResult

Parameter

Type

Description

statusCode

int

Explanation:

HTTP status code.

Value range:

A status code is a group of digits indicating the status of a response. It ranges from 2xx (indicating successes) to 4xx or 5xx (indicating errors).

For more information, see Status Code.

Default value:

None

responseHeaders

Map<String, Object>

Explanation:

Response header list, composed of tuples. In a tuple, the String key indicates the name of the header, and the Object value indicates the value of the header.

Default value:

None

owner

Owner

Explanation:

PFS owner information. For details, see Table 4.

Default value:

None

buckets

List<ObsBucket>

Explanation:

List of PFS information. For details, see Table 13.

Default value:

None

Table 4 Owner

Parameter

Type

Mandatory (Yes/No)

Description

id

String

Yes

Explanation:

Account (domain) ID of the bucket owner.

Value range:

To obtain the account ID, see How Do I Get My Account ID and User ID?

Default value:

None

displayName

String

No

Explanation:

Account name of the owner.

Value range:

To obtain the account name, see How Do I Get My Account ID and User ID?

Default value:

None

Table 5 StorageClassEnum

Constant

Default Value

Description

STANDARD

STANDARD

Standard storage class.

WARM

WARM

Infrequent Access storage class.

COLD

COLD

Archive storage class.

Table 6 ACL

Constant

Description

AccessControlList.REST_CANNED_PRIVATE

Private read/write.

A bucket or object can only be accessed by its owner.

AccessControlList.REST_CANNED_PUBLIC_READ

Public read.

If this permission is granted on a bucket, anyone can read the object list, multipart uploads, bucket metadata, and object versions in the bucket.

If this permission is granted on an object, anyone can read the content and metadata of the object.

AccessControlList.REST_CANNED_PUBLIC_READ_WRITE

Public read/write.

If this permission is granted on a bucket, anyone can read the object list, multipart uploads, and bucket metadata, and can upload or delete objects, initiate multipart uploads, upload parts, assemble parts, copy parts, and cancel multipart upload tasks.

If this permission is granted on an object, anyone can read the content and metadata of the object.

AccessControlList.REST_CANNED_PUBLIC_READ_DELIVERED

Public read on a bucket as well as objects in the bucket.

If this permission is granted on a bucket, anyone can read the object list, multipart tasks, and bucket metadata, and can also read the content and metadata of the objects in the bucket.

This permission cannot be granted on objects.

AccessControlList.REST_CANNED_PUBLIC_READ_WRITE_DELIVERED

Public read/write on a bucket as well as objects in the bucket.

If this permission is granted on a bucket, anyone can read the object list, multipart uploads, and bucket metadata, and can upload or delete objects, initiate multipart upload tasks, upload parts, assemble parts, copy parts, and abort multipart uploads. They can also read the content and metadata of the objects in the bucket.

This permission cannot be granted on objects.

Table 7 AccessControlList

Parameter

Type

Mandatory (Yes/No)

Description

owner

Owner

No

Explanation:

PFS owner information. For details, see Table 4.

delivered

boolean

No

Explanation:

Whether the PFS ACL is applied to all objects in the PFS.

Value range:

true: The PFS ACL is applied to all objects in the PFS.

false: The PFS ACL is not applied to any objects in the PFS.

Default value:

false

grants

Set<GrantAndPermission>

No

Explanation:

Grantee information. For details, see Table 8.

Table 8 GrantAndPermission

Parameter

Type

Mandatory (Yes/No)

Description

grantee

GranteeInterface

Yes

Explanation:

Grantees (users or user groups). For details, see Table 9.

permission

Permission

Yes

Explanation:

Permissions to grant.

Value range:

See Table 12.

Default value:

None

delivered

boolean

No

Explanation:

Whether the PFS ACL is applied to all objects in the PFS.

Value range:

true: The PFS ACL is applied to all objects in the PFS.

false: The PFS ACL is not applied to any objects in the PFS.

Default value:

false

Table 9 GranteeInterface

Parameter

Type

Mandatory (Yes/No)

Description

CanonicalGrantee

CanonicalGrantee

Yes

Explanation:

Grantee (user) information. For details, see Table 10.

GroupGrantee

GroupGrantee

Yes

Explanation:

Grantee (user group) information.

Value range:

See Table 11.

Default value:

None

Table 10 CanonicalGrantee

Parameter

Type

Mandatory (Yes/No)

Description

grantId

String

Yes if Type is set to GranteeUser

Explanation:

Account (domain) ID of the grantee.

Value range:

To obtain the account ID, see How Do I Get My Account ID and User ID?

Default value:

None

displayName

String

No

Parameter description:

Account name of the grantee.

Value range:

To obtain the account name, see How Do I Get My Account ID and User ID?

Default value:

None

Table 11 GroupGrantee

Constant

Description

ALL_USERS

All users.

AUTHENTICATED_USERS

Authorized users. This constant is deprecated.

LOG_DELIVERY

Log delivery group. This constant is deprecated.

Table 12 Permission

Constant

Default Value

Description

PERMISSION_READ

READ

Read permission.

A grantee with this permission for a bucket can obtain the list of objects, multipart uploads, bucket metadata, and object versions in the bucket.

A grantee with this permission for an object can obtain the object content and metadata.

PERMISSION_WRITE

WRITE

Write permission.

A grantee with this permission for a bucket can upload, overwrite, and delete any object or part in the bucket.

This permission is not available for objects.

PERMISSION_READ_ACP

READ_ACP

Permission to read an ACL.

A grantee with this permission can obtain the ACL of a bucket or object.

A bucket or object owner has this permission for their bucket or object by default.

PERMISSION_WRITE_ACP

WRITE_ACP

Permission to modify an ACL.

A grantee with this permission can update the ACL of a bucket or object.

A bucket or object owner has this permission for their bucket or object by default.

This permission allows the grantee to change the access control policies, meaning the grantee has full control over a bucket or object.

PERMISSION_FULL_CONTROL

FULL_CONTROL

Full control access, including read and write permissions for a bucket and its ACL, or for an object and its ACL.

A grantee with this permission for a bucket has READ, WRITE, READ_ACP, and WRITE_ACP permissions for the bucket.

A grantee with this permission for an object has READ, WRITE, READ_ACP, and WRITE_ACP permissions for the object.

Table 13 ObsBucket

Parameter

Type

Description

statusCode

int

Explanation:

HTTP status code.

Value range:

A status code is a group of digits indicating the status of a response. It ranges from 2xx (indicating successes) to 4xx or 5xx (indicating errors).

For more information, see Status Code.

Default value:

None

responseHeaders

Map<String, Object>

Explanation:

Response header list, composed of tuples. In a tuple, the String key indicates the name of the header, and the Object value indicates the value of the header.

Default value:

None

bucketName

String

Explanation:

PFS name.

Restrictions:

  • A PFS name must be unique across all accounts and regions.
  • A PFS name:
    • Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed.
    • Cannot be formatted as an IP address.
    • Cannot start or end with a hyphen (-) or period (.).
    • Cannot contain two consecutive periods (..), for example, my..bucket.
    • Cannot contain periods (.) and hyphens (-) adjacent to each other, for example, my-.bucket or my.-bucket.
  • If you repeatedly create PFSs with the same name in the same region, no error will be reported and the PFS attributes comply with those set in the first creation request.

Default value:

None

owner

Owner

Explanation:

PFS owner information. For details, see Table 4.

creationDate

java.util.Date

Explanation:

Time when the PFS was created.

Default value:

None

location

String

Explanation:

Region where the PFS was created.

Restrictions:

If the specified endpoint is obs.myhuaweicloud.com, this parameter is not required. If any other endpoints are specified, this parameter is required.

Value range:

To learn about valid regions and endpoints, see Regions and Endpoints. An endpoint is the request address for calling an API. Endpoints vary depending on services and regions. To obtain the regions and endpoints, contact the enterprise administrator.

Default value:

If obs.myhuaweicloud.com is used as the endpoint and no region is specified, cn-north-1 (the CN North-Beijing1 region) is used by default.

storageClass

StorageClassEnum

Explanation:

Default storage class. This parameter is not available when creating a PFS. To configure a storage class for the PFS, call an API by referring to Configuring a Storage Class for a Bucket (SDK for Java).

Default value:

None

acl

AccessControlList

Explanation:

An ACL that can be specified at PFS creation. You can use either a pre-defined or a user-defined ACL. For more information about ACLs, see ACLs.

Value range:

  • To use a pre-defined ACL, see Table 6 for the available policies.
  • To use a user-defined ACL, see Table 7 to configure the required parameters.

Default value:

AccessControlList.REST_CANNED_PRIVATE

bucketTypeEnum

BucketTypeEnum

Explanation:

Type of the bucket.

Value range:

PFS

Default value:

None

Code Examples

This example lists parallel file systems.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
import com.obs.services.ObsClient;
import com.obs.services.exception.ObsException;
import com.obs.services.model.BucketTypeEnum;
import com.obs.services.model.ListBucketsRequest;
import com.obs.services.model.ObsBucket;
import java.util.List;
public class ListBucket001 {
    public static void main(String[] args) {
        // Obtain an AK/SK pair using environment variables or import the AK/SK pair in other ways. Using hard coding may result in leakage.
        // Obtain an AK/SK pair on the management console.
        String ak = System.getenv("ACCESS_KEY_ID");
        String sk = System.getenv("SECRET_ACCESS_KEY_ID");
        // (Optional) If you are using a temporary AK/SK pair and a security token to access OBS, you are advised not to use hard coding, which may result in information leakage.
        // Obtain an AK/SK pair and a security token using environment variables or import them in other ways.
        // String securityToken = System.getenv("SECURITY_TOKEN");
        // Enter the endpoint corresponding to the bucket. CN-Hong Kong is used here as an example. Replace it with the one in your actual situation.
        String endPoint = "https://obs.ap-southeast-1.myhuaweicloud.com";
        // Obtain an endpoint using environment variables or import it in other ways.
        //String endPoint = System.getenv("ENDPOINT");
        
        // Create an ObsClient instance.
        // Use the permanent AK/SK pair to initialize the client.
        ObsClient obsClient = new ObsClient(ak, sk,endPoint);
        // Use the temporary AK/SK pair and security token to initialize the client.
        // ObsClient obsClient = new ObsClient(ak, sk, securityToken, endPoint);

        try {
            // List parallel file systems.
            ListBucketsRequest request = new ListBucketsRequest();
            request.setBucketType(BucketTypeEnum.PFS);
            List<ObsBucket> buckets = obsClient.listBuckets(request);
            for (ObsBucket bucket : buckets) {
                System.out.println("BucketName:" + bucket.getBucketName());
                System.out.println("CreationDate:" + bucket.getCreationDate());
                System.out.println("Location:" + bucket.getLocation());
            }
            System.out.println("listBuckets successfully");
        } catch (ObsException e) {
            System.out.println("listBuckets failed");
            // Request failed. Print the HTTP status code.
            System.out.println("HTTP Code:" + e.getResponseCode());
            // Request failed. Print the server-side error code.
            System.out.println("Error Code:" + e.getErrorCode());
            // Request failed. Print the error details.
            System.out.println("Error Message:" + e.getErrorMessage());
            // Request failed. Print the request ID.
            System.out.println("Request ID:" + e.getErrorRequestId());
            System.out.println("Host ID:" + e.getErrorHostId());
            e.printStackTrace();
        } catch (Exception e) {
            System.out.println("listBuckets failed");
            // Print other error information.
            e.printStackTrace();
        }
    }
}