Help Center/ Object Storage Service/ SDK Reference/ Java/ Parallel File Systems (SDK for Java)/ Creating a Parallel File System (SDK for Java)
Updated on 2024-06-18 GMT+08:00

Creating a Parallel File System (SDK for Java)

Function

Parallel File System (PFS), a sub-product of OBS, is a high-performance file system with only milliseconds of latency. PFS supports TB/s bandwidth and millions of IOPS, which make it ideal for processing high-performance computing (HPC) workloads.

You can access data in a PFS using standard OBS APIs. You can also read and write data using obsfs, a PFS client that supports POSIX. obsfs can be deployed on a Linux ECS. Then, you can use obsfs to mount a PFS to the ECS and perform a series of operations online, including creating, deleting, and renaming files or directories, and modifying files.

This API creates a PFS.

If you have any questions during development, post them on the Issues page of GitHub.

Restrictions

  • To create a PFS, you must have the obs:bucket:CreateBucket permission. IAM is recommended for granting permissions. For details, see IAM Custom Policies.
  • The mapping between OBS regions and endpoints must comply with what is listed in Regions and Endpoints.
  • An existing bucket cannot be changed to a PFS.
  • PFSs do not support quota configuration. By default, there is no quota limit.
  • A maximum of 100 buckets and parallel file systems in total can be created globally for an account. There is no limit on the number or size of objects stored in a bucket.
  • The name of a PFS must be unique in OBS. If you repeatedly create PFSs with the same name in the same region, an HTTP status code 200 will be returned. In other cases, creating a PFS with an existing PFS name will have an HTTP status code 409 returned, indicating that such a PFS already exists.
  • The name of a deleted PFS can be reused for another bucket or PFS about 30 minutes after the deletion.
  • Not all regions support the creation of multi-AZ PFSs. You can check whether a region allows you to create multi-AZ PFSs by referring to Product Pricing Details.

Method

obsClient.createBucket(CreateBucketRequest request)

Request Parameters

Table 1 List of request parameters

Parameter

Type

Mandatory (Yes/No)

Description

request

CreateBucketRequest

Yes

Explanation:

Request parameters for creating a PFS. For details, see Table 2.

Table 2 CreateBucketRequest

Parameter

Type

Mandatory (Yes/No)

Description

bucketName

String

Yes

Explanation:

PFS name.

Restrictions:

  • A PFS name must be unique across all accounts and regions.
  • A PFS name:
    • Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed.
    • Cannot be formatted as an IP address.
    • Cannot start or end with a hyphen (-) or period (.).
    • Cannot contain two consecutive periods (..), for example, my..bucket.
    • Cannot contain periods (.) and hyphens (-) adjacent to each other, for example, my-.bucket or my.-bucket.
  • If you repeatedly create PFSs with the same name in the same region, no error will be reported and the PFS attributes comply with those set in the first creation request.

Default value:

None

location

String

Yes if the region where the OBS service resides is not the default region

Explanation:

Region where the PFS is to be created.

Restrictions:

If the specified endpoint is obs.myhuaweicloud.com, this parameter is not required. If any other endpoints are specified, this parameter is required.

Value range:

To learn about valid regions and endpoints, see Regions and Endpoints. An endpoint is the request address for calling an API. Endpoints vary depending on services and regions. To obtain the regions and endpoints, contact the enterprise administrator.

Default value:

If obs.myhuaweicloud.com is used as the endpoint and no region is specified, cn-north-1 (the CN North-Beijing1 region) is used by default.

acl

AccessControlList

No

Explanation:

An ACL that can be specified at PFS creation. You can use either a pre-defined or a user-defined ACL. For more information about ACLs, see ACLs.

Value range:

  • To use a pre-defined ACL, see Table 3 for the available policies.
  • To use a user-defined ACL, see Table 5 to configure the required parameters.

Default value:

AccessControlList.REST_CANNED_PRIVATE

storageClass

StorageClassEnum

No

Explanation:

Default storage class. This parameter is not available when creating a PFS. To configure a storage class for the PFS, call an API by referring to Configuring a Storage Class for a Bucket (SDK for Java).

Default value:

None

extensionPermissionMap

Map<ExtensionBucketPermissionEnum, Set<String>>

No

Explanation:

An ACL for granting PFS permissions to one or more accounts. ExtensionBucketPermissionEnum specifies the permissions to grant, and Set<String> describes the list of account IDs (indicated by domain_id) the granted permissions apply to.

Value range:

Default value:

None

epid

String

No

Explanation:

Enterprise project ID that can be specified at PFS creation. If you have enabled EPS, you can obtain the project ID from the EPS console.

Restrictions:

The value of epid is a UUID. epid is not required if you have not enabled EPS yet.

Example: 9892d768-2d13-450f-aac7-ed0e44c2585f

Default value:

None

availableZone

AvailableZoneEnum

No

Explanation:

Data redundancy type that can be specified at PFS creation.

Restrictions:

Multi-AZ redundancy does not support Archive storage. If the region where the PFS is located does not support multi-AZ storage, the PFS adopts single-AZ storage by default.

Value range:

To configure multi-AZ storage for the PFS, set this parameter to MULTI_AZ. To configure single-AZ storage (default value assigned by OBS) for the PFS, you do not need to specify this parameter.

Default value:

If this parameter is left blank, single AZ is specified by default.

bucketType

BucketTypeEnum

No

Explanation:

Type of the bucket to be created.

Value range:

PFS

Default value:

None

Table 3 ACL

Constant

Description

AccessControlList.REST_CANNED_PRIVATE

Private read/write.

A bucket or object can only be accessed by its owner.

AccessControlList.REST_CANNED_PUBLIC_READ

Public read.

If this permission is granted on a bucket, anyone can read the object list, multipart uploads, bucket metadata, and object versions in the bucket.

If this permission is granted on an object, anyone can read the content and metadata of the object.

AccessControlList.REST_CANNED_PUBLIC_READ_WRITE

Public read/write.

If this permission is granted on a bucket, anyone can read the object list, multipart uploads, and bucket metadata, and can upload or delete objects, initiate multipart uploads, upload parts, assemble parts, copy parts, and cancel multipart upload tasks.

If this permission is granted on an object, anyone can read the content and metadata of the object.

AccessControlList.REST_CANNED_PUBLIC_READ_DELIVERED

Public read on a bucket as well as objects in the bucket.

If this permission is granted on a bucket, anyone can read the object list, multipart tasks, and bucket metadata, and can also read the content and metadata of the objects in the bucket.

This permission cannot be granted on objects.

AccessControlList.REST_CANNED_PUBLIC_READ_WRITE_DELIVERED

Public read/write on a bucket as well as objects in the bucket.

If this permission is granted on a bucket, anyone can read the object list, multipart uploads, and bucket metadata, and can upload or delete objects, initiate multipart upload tasks, upload parts, assemble parts, copy parts, and abort multipart uploads. They can also read the content and metadata of the objects in the bucket.

This permission cannot be granted on objects.

Table 4 ExtensionBucketPermissionEnum

Constant

Description

GRANT_READ

Grants the READ permission to an account ID.

The account with the READ permission can list objects, multipart uploads, and object versions in the bucket you are creating, and can obtain bucket metadata.

GRANT_WRITE

Grants the WRITE permission to an account ID.

The account with the WRITE permission can create, delete, and overwrite objects in the bucket you are creating, and can initiate or abort multipart uploads, as well as upload, copy, and assemble parts.

GRANT_READ_ACP

Grants the READ_ACP permission to an account ID.

The account with the READ_ACP permission can read the ACL of the bucket you are creating.

GRANT_WRITE_ACP

Grants the WRITE_ACP permission to an account ID.

The account with the WRITE_ACP permission can modify the ACL of the bucket you are creating.

GRANT_FULL_CONTROL

Grants the FULL_CONTROL permission to an account ID.

The account with the FULL_CONTROL permission can perform any operation on the bucket you are creating.

GRANT_READ_DELIVERED

Grants the READ permission to an account ID. By default, this READ permission applies to all objects in the bucket.

GRANT_FULL_CONTROL_DELIVERED

Grants the FULL_CONTROL permission to an account ID. By default, this FULL_CONTROL permission applies to all objects in the bucket.

Table 5 AccessControlList

Parameter

Type

Mandatory (Yes/No)

Description

owner

Owner

No

Explanation:

PFS owner information. For details, see Table 6.

delivered

boolean

No

Explanation:

Whether the PFS ACL is applied to all objects in the PFS.

Value range:

true: The PFS ACL is applied to all objects in the PFS.

false: The PFS ACL is not applied to any objects in the PFS.

Default value:

false

grants

Set<GrantAndPermission>

No

Explanation:

Grantee information. For details, see Table 7.

Table 6 Owner

Parameter

Type

Mandatory (Yes/No)

Description

id

String

Yes

Explanation:

Account (domain) ID of the bucket owner.

Value range:

To obtain the account ID, see How Do I Get My Account ID and User ID?

Default value:

None

displayName

String

No

Explanation:

Account name of the owner.

Value range:

To obtain the account name, see How Do I Get My Account ID and User ID?

Default value:

None

Table 7 GrantAndPermission

Parameter

Type

Mandatory (Yes/No)

Description

grantee

GranteeInterface

Yes

Explanation:

Grantees (users or user groups). For details, see Table 8.

permission

Permission

Yes

Explanation:

Permissions to grant.

Value range:

See Table 11.

Default value:

None

delivered

boolean

No

Explanation:

Whether the PFS ACL is applied to all objects in the PFS.

Value range:

true: The PFS ACL is applied to all objects in the PFS.

false: The PFS ACL is not applied to any objects in the PFS.

Default value:

false

Table 8 GranteeInterface

Parameter

Type

Mandatory (Yes/No)

Description

CanonicalGrantee

CanonicalGrantee

Yes

Explanation:

Grantee (user) information. For details, see Table 9.

GroupGrantee

GroupGrantee

Yes

Explanation:

Grantee (user group) information.

Value range:

See Table 10.

Default value:

None

Table 9 CanonicalGrantee

Parameter

Type

Mandatory (Yes/No)

Description

grantId

String

Yes if Type is set to GranteeUser

Explanation:

Account (domain) ID of the grantee.

Value range:

To obtain the account ID, see How Do I Get My Account ID and User ID?

Default value:

None

displayName

String

No

Parameter description:

Account name of the grantee.

Value range:

To obtain the account name, see How Do I Get My Account ID and User ID?

Default value:

None

Table 10 GroupGrantee

Constant

Description

ALL_USERS

All users.

AUTHENTICATED_USERS

Authorized users. This constant is deprecated.

LOG_DELIVERY

Log delivery group. This constant is deprecated.

Table 11 Permission

Constant

Default Value

Description

PERMISSION_READ

READ

Read permission.

A grantee with this permission for a bucket can obtain the list of objects, multipart uploads, bucket metadata, and object versions in the bucket.

A grantee with this permission for an object can obtain the object content and metadata.

PERMISSION_WRITE

WRITE

Write permission.

A grantee with this permission for a bucket can upload, overwrite, and delete any object or part in the bucket.

This permission is not available for objects.

PERMISSION_READ_ACP

READ_ACP

Permission to read an ACL.

A grantee with this permission can obtain the ACL of a bucket or object.

A bucket or object owner has this permission for their bucket or object by default.

PERMISSION_WRITE_ACP

WRITE_ACP

Permission to modify an ACL.

A grantee with this permission can update the ACL of a bucket or object.

A bucket or object owner has this permission for their bucket or object by default.

This permission allows the grantee to change the access control policies, meaning the grantee has full control over a bucket or object.

PERMISSION_FULL_CONTROL

FULL_CONTROL

Full control access, including read and write permissions for a bucket and its ACL, or for an object and its ACL.

A grantee with this permission for a bucket has READ, WRITE, READ_ACP, and WRITE_ACP permissions for the bucket.

A grantee with this permission for an object has READ, WRITE, READ_ACP, and WRITE_ACP permissions for the object.

Table 12 StorageClassEnum

Constant

Default Value

Description

STANDARD

STANDARD

Standard storage class.

WARM

WARM

Infrequent Access storage class.

COLD

COLD

Archive storage class.

Table 13 AvailableZoneEnum

Constant

Default Value

Description

MULTI_AZ

3az

Multi-AZ redundancy

Responses

Table 14 response

Parameter

Type

Description

statusCode

int

Explanation:

HTTP status code.

Value range:

A status code is a group of digits indicating the status of a response. It ranges from 2xx (indicating successes) to 4xx or 5xx (indicating errors).

For more information, see Status Code.

Default value:

None

responseHeaders

Map<String, Object>

Explanation:

Response header list, composed of tuples. In a tuple, the String key indicates the name of the header, and the Object value indicates the value of the header.

Default value:

None

bucketName

String

Explanation:

PFS name.

Restrictions:

  • A PFS name must be unique across all accounts and regions.
  • A PFS name:
    • Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed.
    • Cannot be formatted as an IP address.
    • Cannot start or end with a hyphen (-) or period (.).
    • Cannot contain two consecutive periods (..), for example, my..bucket.
    • Cannot contain periods (.) and hyphens (-) adjacent to each other, for example, my-.bucket or my.-bucket.
  • If you repeatedly create PFSs with the same name in the same region, no error will be reported and the PFS attributes comply with those set in the first creation request.

Default value:

None

owner

Owner

Explanation:

PFS owner information. For details, see Table 6.

creationDate

java.util.Date

Explanation:

Time when the PFS was created.

Default value:

None

location

String

Explanation:

Region where the PFS was created.

Restrictions:

If the specified endpoint is obs.myhuaweicloud.com, this parameter is not required. If any other endpoints are specified, this parameter is required.

Value range:

To learn about valid regions and endpoints, see Regions and Endpoints. An endpoint is the request address for calling an API. Endpoints vary depending on services and regions. To obtain the regions and endpoints, contact the enterprise administrator.

Default value:

If obs.myhuaweicloud.com is used as the endpoint and no region is specified, cn-north-1 (the CN North-Beijing1 region) is used by default.

storageClass

StorageClassEnum

Explanation:

Default storage class. This parameter is not available when creating a PFS. To configure a storage class for the PFS, call an API by referring to Configuring a Storage Class for a Bucket (SDK for Java).

Default value:

None

acl

AccessControlList

Explanation:

An ACL that can be specified at PFS creation. You can use either a pre-defined or a user-defined ACL. For more information about ACLs, see ACLs.

Value range:

  • To use a pre-defined ACL, see Table 3 for the available policies.
  • To use a user-defined ACL, see Table 5 to configure the required parameters.

Default value:

AccessControlList.REST_CANNED_PRIVATE

bucketTypeEnum

BucketTypeEnum

Explanation:

Type of the bucket.

Value range:

PFS

Default value:

None

Code Examples

This example creates parallel file system examplebucket.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
import com.obs.services.ObsClient;
import com.obs.services.exception.ObsException;
import com.obs.services.model.BucketTypeEnum;
import com.obs.services.model.CreateBucketRequest;
import com.obs.services.model.HeaderResponse;
public class CreateBucket001 {
    public static void main(String[] args) {
        // Obtain an AK/SK pair using environment variables or import the AK/SK pair in other ways. Using hard coding may result in leakage.
        // Obtain an AK/SK pair on the management console.
        String ak = System.getenv("ACCESS_KEY_ID");
        String sk = System.getenv("SECRET_ACCESS_KEY_ID");
        // (Optional) If you are using a temporary AK/SK pair and a security token to access OBS, you are advised not to use hard coding, which may result in information leakage.
        // Obtain an AK/SK pair and a security token using environment variables or import them in other ways.
        // String securityToken = System.getenv("SECURITY_TOKEN");
        // Enter the endpoint corresponding to the bucket. CN-Hong Kong is used here as an example. Replace it with the one in your actual situation.
        String endPoint = "https://obs.ap-southeast-1.myhuaweicloud.com";
        // Obtain an endpoint using environment variables or import it in other ways.
        //String endPoint = System.getenv("ENDPOINT");
        
        // Create an ObsClient instance.
        // Use the permanent AK/SK pair to initialize the client.
        ObsClient obsClient = new ObsClient(ak, sk,endPoint);
        // Use the temporary AK/SK pair and security token to initialize the client.
        // ObsClient obsClient = new ObsClient(ak, sk, securityToken, endPoint);

        try {
            // The parallel file system is successfully created.
            CreateBucketRequest request = new CreateBucketRequest();
            request.setBucketName("examplebucket");
            request.setBucketType(BucketTypeEnum.PFS);
            // Set the location. location must match the endpoint.
            request.setLocation("your_region");
            HeaderResponse response = obsClient.createBucket(request);
            System.out.println("CreateBucket successfully");
            System.out.println("StatusCode:" + response.getStatusCode());
            System.out.println("RequestId:" + response.getRequestId());
        } catch (ObsException e) {
            System.out.println("CreateBucket failed");
            // Request failed. Print the HTTP status code.
            System.out.println("HTTP Code:" + e.getResponseCode());
            // Request failed. Print the server-side error code.
            System.out.println("Error Code:" + e.getErrorCode());
            // Request failed. Print the error details.
            System.out.println("Error Message:" + e.getErrorMessage());
            // Request failed. Print the request ID.
            System.out.println("Request ID:" + e.getErrorRequestId());
            System.out.println("Host ID:" + e.getErrorHostId());
            e.printStackTrace();
        } catch (Exception e) {
            System.out.println("CreateBucket failed");
            // Print other error information.
            e.printStackTrace();
        }
    }
}