Creating a Parallel File System

Function

Parallel File System (PFS), a sub-product of OBS, is a high-performance file system with only milliseconds of latency. PFS supports TB/s bandwidth and millions of IOPS, which make it ideal for processing high-performance computing (HPC) workloads.

You can access data in a PFS using standard OBS APIs. You can also read and write data using obsfs, a PFS client that supports POSIX. obsfs can be deployed on a Linux ECS. Then, you can use obsfs to mount a PFS to the ECS and perform a series of operations online, including creating, deleting, and renaming files or directories, and modifying files.

This API creates a PFS.

Restrictions

To create a PFS, you must have the obs:bucket:CreateBucket permission. IAM is recommended for granting permissions. For details, see IAM Custom Policies.
The mapping between OBS regions and endpoints must comply with what is listed in Regions and Endpoints.

An existing bucket cannot be changed to a PFS.
PFSs do not support quota configuration. By default, there is no quota limit.
A maximum of 100 buckets and parallel file systems in total can be created globally for an account. There is no limit on the number or size of objects stored in a bucket.
The name of a PFS must be unique in OBS. If you repeatedly create PFSs with the same name in the same region, an HTTP status code 200 will be returned. In other cases, creating a PFS with an existing PFS name will have an HTTP status code 409 returned, indicating that such a PFS already exists.
The name of a deleted PFS can be reused for another bucket or PFS about 30 minutes after the deletion.
Not all regions support the creation of multi-AZ PFSs. You can check whether a region allows you to create multi-AZ PFSs by referring to Product Pricing Details.

Method

obsClient.createBucket(CreateBucketRequest request)

Request Parameters

**Table 1** List of request parameters
Parameter	Type	Mandatory (Yes/No)	Description
request	CreateBucketRequest	Yes	Explanation: Request parameters for creating a PFS. For details, see Table 2.

**Table 2** CreateBucketRequest
Parameter	Type	Mandatory (Yes/No)	Description
bucketName	String	Yes	Explanation: PFS name. Restrictions: A PFS name must be unique across all accounts and regions. A PFS name: Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed. Cannot be formatted as an IP address. Cannot start or end with a hyphen (-) or period (.) Cannot contain two consecutive periods (.), for example, my..bucket. Cannot contain a period (.) and a hyphen (-) adjacent to each other, for example, my-.bucket or my.-bucket. If you repeatedly create PFSs with the same name in the same region, no error will be reported and the PFS properties comply with those set in the first creation request. Default value: None
location	String	Yes if the OBS service region is not the default region	Explanation: Region where the PFS is located. Restrictions: If the specified endpoint is obs.myhuaweicloud.com, this parameter is not required. If any other endpoints are specified, this parameter is required. Value range: To learn about valid regions and endpoints, see Regions and Endpoints. An endpoint is the request address for calling an API. Endpoints vary depending on services and regions. To obtain the regions and endpoints, contact the enterprise administrator. Default value: If obs.myhuaweicloud.com is used as the endpoint and no region is specified, cn-north-1 (the CN North-Beijing1 region) is used by default.
acl	AccessControlList	No	Explanation: An ACL that can be specified at PFS creation. You can use either a pre-defined or a user-defined ACL. For more information about ACLs, see ACLs. Value range: To use a pre-defined ACL, see Table 3 for the available options. To use a user-defined ACL, see Table 5 to configure the required parameters. Default value: AccessControlList.REST_CANNED_PRIVATE
storageClass	StorageClassEnum	No	Explanation: When creating a PFS, you cannot configure its default storage class. Default value: None
extensionPermissionMap	Map<ExtensionBucketPermissionEnum, Set<String>>	No	Explanation: Authorization map of the PFS ACL. You can grant PFS permissions to one or more accounts. ExtensionBucketPermissionEnum in the map specifies the permissions to grant, and Set<String> in the map describes the list of account IDs (indicated by domain_id) the granted permissions apply to. Value range: For details about the available permissions, see Table 4. Default value: None
epid	String	No	Explanation: Enterprise project ID that can be specified during PFS creation. If you have enabled EPS, you can obtain the project ID from the EPS console. Restrictions: The value of epid is a UUID. epid is not required if you have not enabled EPS yet. Example: 9892d768-2d13-450f-aac7-ed0e44c2585f Default value: None
availableZone	AvailableZoneEnum	No	Explanation: Data redundancy type that can be specified during PFS creation. Restrictions: Multi-AZ redundancy is not available for the Archive storage. If the region where the PFS is located does not support multi-AZ storage, the PFS adopts single-AZ storage by default. Value range: To configure multi-AZ storage for the PFS, set this parameter to MULTI_AZ. To configure single-AZ storage (default value assigned by OBS) for the PFS, you do not need to specify this parameter. Default value: If this parameter is left blank, single-AZ storage is used by default.
bucketType	BucketTypeEnum	No	Explanation: Type of the bucket. Value range: PFS Default value: None

**Table 3** Pre-defined ACL
Constant	Description
AccessControlList.REST_CANNED_PRIVATE	Private read and write. A bucket or an object can only be accessed by its owner.
AccessControlList.REST_CANNED_PUBLIC_READ	Public read. If this permission is set for a bucket, everyone can obtain the list of objects, multipart uploads, and object versions in the bucket, as well as metadata of the bucket. If this permission is set for an object, everyone can obtain the content and metadata of the object.
AccessControlList.REST_CANNED_PUBLIC_READ_WRITE	Public read and write. If this permission is set for a bucket, everyone can obtain the object list, multipart uploads, and bucket metadata, and can upload or delete objects, initiate multipart uploads, upload parts, assemble parts, copy parts, and abort multipart uploads. If this permission is set for an object, everyone can obtain the content and metadata of the object.
AccessControlList.REST_CANNED_PUBLIC_READ_DELIVERED	Public read on a bucket as well as the objects in the bucket. If this permission is set for a bucket, everyone can obtain the object list, multipart uploads, and bucket metadata in the bucket, and obtain the content and metadata of the objects in the bucket. This permission cannot be set for objects.
AccessControlList.REST_CANNED_PUBLIC_READ_WRITE_DELIVERED	Public read and write on a bucket as well as the objects in the bucket. If this permission is set for a bucket, everyone can obtain the object list, multipart uploads, and bucket metadata, and can upload or delete objects, initiate multipart uploads, upload parts, assemble parts, copy parts, abort multipart uploads, and obtain the content and metadata of objects in the bucket. This permission cannot be set for objects.
AccessControlList.REST_CANNED_BUCKET_OWNER_FULL_CONTROL	If this permission is set for an object, only the bucket and object owners have the full control over the object. By default, if you upload an object to a bucket of any other user, the bucket owner does not have the permissions on your object. After you grant this permission to the bucket owner, the bucket owner can have full control over your object. For example, if user A uploads object x to user B's bucket, user B does not have the control over object x. If user A sets bucket-owner-full-control for object x, user B then has the control over object x.

**Table 4** ExtensionBucketPermissionEnum
Constant	Description
GRANT_READ	Grants the READ permission to an account ID. The account with the READ permission can list objects, multipart uploads, and object versions in the bucket you are creating, as well as obtain bucket metadata.
GRANT_WRITE	Grants the WRITE permission to an account ID. The account with the WRITE permission can create, delete, and overwrite objects in the bucket you are creating, and can initiate or abort multipart uploads, as well as upload, copy, and assemble parts.
GRANT_READ_ACP	Grants the READ_ACP permission to an account ID. The account with the READ_ACP permission can read the ACL of the bucket you are creating.
GRANT_WRITE_ACP	Grants the WRITE_ACP permission to an account ID. The account with the WRITE_ACP permission can modify the ACL of the bucket you are creating.
GRANT_FULL_CONTROL	Grants the FULL_CONTROL permission to an account ID. The account with the FULL_CONTROL permission can perform any operation on the bucket you are creating.
GRANT_READ_DELIVERED	Grants the READ permission to an account ID. By default, this READ permission applies to all objects in the bucket.
GRANT_FULL_CONTROL_DELIVERED	Grants the FULL_CONTROL permission to an account ID. By default, this FULL_CONTROL permission applies to all objects in the bucket.

**Table 5** AccessControlList
Parameter	Type	Mandatory (Yes/No)	Description
owner	Owner	No	Explanation: PFS owner information. For details, see Table 6.
delivered	boolean	No	Explanation: Whether the PFS ACL is applied to all objects in the PFS. Value range: true: The PFS ACL is applied to all objects in the PFS. false: The PFS ACL is not applied to any objects in the PFS. Default value: false
grants	Set<GrantAndPermission>	No	Explanation: Grantee information. For details, see Table 7.

**Table 6** Owner
Parameter	Type	Mandatory (Yes/No)	Description
id	String	Yes	Explanation: Account (domain) ID of the bucket owner. Default value: None
displayName	String	No	Explanation: Account name of the owner. Default value: None

**Table 7** GrantAndPermission
Parameter	Type	Mandatory (Yes/No)	Description
grantee	CanonicalGrantee or GroupGrantee	Yes	Explanation: Grantees (users or user groups). Value range: Grantee (user) information. For details, see Table 8. Grantee (user group) information. For details, see Table 9.
permission	Permission	Yes	Explanation: Permissions to grant. Value range: See Table 10. Default value: None
delivered	boolean	No	Explanation: Whether the PFS ACL is applied to all objects in the PFS. Value range: true: The PFS ACL is applied to all objects in the PFS. false: The PFS ACL is not applied to any objects in the PFS. Default value: false

**Table 8** CanonicalGrantee
Parameter	Type	Mandatory (Yes/No)	Description
grantId	String	Yes if Type is set to a user	Explanation: Account (domain) ID of the grantee. Default value: None
displayName	String	No	Explanation: Account name of the grantee. Default value: None

**Table 9** GroupGrantee
Constant	Description
ALL_USERS	All users.
AUTHENTICATED_USERS	Authorized users. This constant is deprecated.
LOG_DELIVERY	Log delivery group. This constant is deprecated.

**Table 10** Permission
Constant	Original Value	Description
PERMISSION_READ	READ	Read permission. A grantee with this permission for a bucket can obtain the list of objects, multipart uploads, and object versions in the bucket, as well as metadata of the bucket. A grantee with this permission for an object can obtain the object content and metadata.
PERMISSION_WRITE	WRITE	Write permission. A grantee with this permission for a bucket can upload, overwrite, and delete any object or part in the bucket. This permission is not available for objects.
PERMISSION_READ_ACP	READ_ACP	Permission to read an ACL. A grantee with this permission can obtain the ACL of a bucket or object. A bucket or object owner has this permission for the bucket or object permanently.
PERMISSION_WRITE_ACP	WRITE_ACP	Permission to modify an ACL. A grantee with this permission can update the ACL of a bucket or object. A bucket or object owner has this permission for the bucket or object permanently. This permission allows the grantee to change the access control policies, meaning the grantee has full control over a bucket or object.
PERMISSION_FULL_CONTROL	FULL_CONTROL	Full control access, including read and write permissions for a bucket and its ACL, or for an object and its ACL. A grantee with this permission for a bucket has READ, WRITE, READ_ACP, and WRITE_ACP permissions for the bucket. A grantee with this permission for an object has READ, READ_ACP, and WRITE_ACP permissions for the object.

**Table 11** StorageClassEnum
Constant	Original Value	Description
STANDARD	STANDARD	Standard storage class.
WARM	WARM	Infrequent Access storage class.
COLD	COLD	Archive storage class.

**Table 12** AvailableZoneEnum
Constant	Original Value	Description
MULTI_AZ	3az	Multi-AZ redundancy

Responses

**Table 13** response
Parameter	Type	Description
statusCode	int	Explanation: HTTP status code. Value range: A status code is a group of digits indicating the status of a response. It ranges from 2xx (indicating successes) to 4xx or 5xx (indicating errors). For more information, see Status Code. Default value: None
responseHeaders	Map<String, Object>	Explanation: Response header list, composed of tuples. In a tuple, the String key indicates the name of the header, and the Object value indicates the value of the header. Default value: None
bucketName	String	Explanation: PFS name. Restrictions: A PFS name must be unique across all accounts and regions. A PFS name: Must be 3 to 63 characters long and start with a digit or letter. Lowercase letters, digits, hyphens (-), and periods (.) are allowed. Cannot be formatted as an IP address. Cannot start or end with a hyphen (-) or period (.) Cannot contain two consecutive periods (.), for example, my..bucket. Cannot contain a period (.) and a hyphen (-) adjacent to each other, for example, my-.bucket or my.-bucket. If you repeatedly create PFSs with the same name in the same region, no error will be reported and the PFS properties comply with those set in the first creation request. Default value: None
owner	Owner	Explanation: PFS owner information. For details, see Table 6.
creationDate	java.util.Date	Explanation: Time when the PFS was created. Default value: None
location	String	Explanation: Region where the PFS is located. Restrictions: If the specified endpoint is obs.myhuaweicloud.com, this parameter is not required. If any other endpoints are specified, this parameter is required. Value range: To learn about valid regions and endpoints, see Regions and Endpoints. An endpoint is the request address for calling an API. Endpoints vary depending on services and regions. To obtain the regions and endpoints, contact the enterprise administrator. Default value: If obs.myhuaweicloud.com is used as the endpoint and no region is specified, cn-north-1 (the CN North-Beijing1 region) is used by default.
storageClass	StorageClassEnum	Explanation: When creating a PFS, you cannot configure its default storage class. Default value: None
acl	AccessControlList	Explanation: An ACL that can be specified at PFS creation. You can use either a pre-defined or a user-defined ACL. For more information about ACLs, see ACLs. Value range: To use a pre-defined ACL, see Table 3 for the available options. To use a user-defined ACL, see Table 5 to configure the required parameters. Default value: AccessControlList.REST_CANNED_PRIVATE
bucketTypeEnum	BucketTypeEnum	Explanation: Type of the bucket. Value range: PFS Default value: None

Sample Code

This example creates a PFS named examplebucket.

import android.util.Log;
import com.obs.services.ObsClient;
import com.obs.services.exception.ObsException;
import com.obs.services.model.CreateVirtualBucketRequest;
import com.obs.services.model.CreateVirtualBucketResult;

// Note: Network operations must be performed in a child thread. Otherwise, NetworkOnMainThreadException will be thrown.
new Thread(() -> {
    // Security warning: Hard-coded AK and SK are risky. You are advised to obtain the AK and SK from the security credential management system.
    String ak = System.getenv("ACCESS_KEY_ID");
    String sk = System.getenv("SECRET_ACCESS_KEY");
    String endPoint = "https://your-endpoint";

    ObsClient obsClient = null;
    try {
        obsClient = new ObsClient(ak, sk, endPoint);

        // Create a PFS.
        CreateVirtualBucketRequest request = new CreateVirtualBucketRequest();
        request.setBucketName("examplebucket");
        request.setLocation("cn-north-4");

        CreateVirtualBucketResult result = obsClient.createVirtualBucket(request);
        Log.i("ObsDemo", "Succeeded in creating a PFS. Bucket name: " + result.getBucketName()
                + ". Region: " + result.getLocation()
                + ". statusCode: " + result.getStatusCode());
    } catch (ObsException e) {
        Log.e("ObsDemo", "Failed to create a PFS. HTTP status code: " + e.getResponseCode()
                + ". Error code: " + e.getErrorCode()
                + ". Error message: " + e.getErrorMessage());
    } finally {
        if (obsClient != null) {
            try {
                obsClient.close();
            } catch (Exception e) {
                Log.e("ObsDemo", "Failed to close ObsClient.", e);
            }
        }
    }
}).start();

Parent topic: Parallel File Systems

Previous topic: Parallel File Systems

Next topic: Listing Parallel File Systems