Help Center/ Object Storage Service/ User Guide/ Parallel File System/ Parallel File System Features
Updated on 2026-04-23 GMT+08:00
View PDF
Share

Parallel File System Features

Compatibility Between OBS Bucket APIs and Parallel File Systems

You can call some OBS bucket APIs to use parallel file systems. There may be additional requirements when you call these APIs.

For details about the OBS bucket APIs, see Object Storage Service API Reference.

APIs for Basic Bucket Operations

Table 1 APIs for basic bucket operations

API

Compatible with Parallel File Systems

Differences

Listing buckets

Yes

The x-obs-bucket-type:POSIX header is required for obtaining the list of parallel file systems.

Creating a bucket

Yes

The x-obs-fs-file-interface:Enabled header is required for creating a parallel file system.

Listing objects in a bucket

Yes

-

Obtaining bucket metadata

Yes

-

Obtaining location of a bucket

Yes

-

Deleting a bucket

Yes

-

APIs for Advanced Bucket Settings

Table 2 APIs for advanced bucket settings

API

Compatible with Parallel File Systems

Differences

Configuring a bucket policy

Yes

-

Obtaining a bucket policy

Yes

-

Deleting a bucket policy

Yes

-

Configuring a bucket ACL

Yes

-

Obtaining bucket ACL information

Yes

-

Configuring logging for a bucket

Yes

-

Obtaining a bucket logging configuration

Yes

-

Configuring bucket lifecycle rules

Yes

See Lifecycle Management

Obtaining bucket lifecycle rules

Yes

-

Deleting bucket lifecycle rules

Yes

-

Configuring versioning for a bucket

No

-

Obtaining bucket versioning status

No

-

Configuring event notification for a bucket

Yes

-

Obtaining the event notification configuration of a bucket

Yes

-

Configuring storage class for a bucket

No

-

Obtaining bucket storage class information

No

-

Configuring cross-region replication for a bucket

No

-

Obtaining the cross-region replication configuration of a bucket

No

-

Deleting the cross-region replication configuration of a bucket

No

-

Configuring tags for a bucket

Yes

-

Obtaining bucket tags

Yes

-

Deleting bucket tags

Yes

-

Configuring bucket storage quota

Yes

-

Querying bucket storage quota

Yes

-

Querying information about used space in a bucket

Yes

-

Configuring bucket inventories

No

-

Obtaining bucket inventories

No

-

Listing bucket inventories

No

-

Deleting bucket inventories

No

-

Configuring a custom domain name for a bucket

Yes

-

Obtaining the custom domain name of a bucket

Yes

-

Deleting a custom domain name of a bucket

Yes

-

Configuring bucket encryption

No

-

Obtaining bucket encryption configuration

No

-

Deleting the encryption configuration of a bucket

No

-

Configuring direct reading for Archive objects in a bucket

Yes

-

Obtaining direct reading configuration for Archive objects in a bucket

Yes

-

Deleting direct reading configuration of Archive objects in a bucket

Yes

-

Configuring mirroring-based back-to-source rules

No

-

Obtaining back-to-source by mirroring rules

No

-

Deleting back-to-source by mirroring rules

No

-

Configuring online decompression policies

No

-

Obtaining online decompression policies

No

-

Deleting online decompression policies

No

-

Configuring a default WORM policy for a bucket

No

-

Obtaining the default WORM policy of a bucket

No

-

Configuring static website hosting for a bucket

No

-

Obtaining the static website hosting configuration of a bucket

No

-

Deleting the static website hosting configuration of a bucket

No

-

Configuring bucket CORS

Yes

-

Obtaining the CORS configuration of a bucket

Yes

-

Deleting the CORS configuration of a bucket

Yes

-

Sending an OPTIONS request to a bucket

No

-

Sending an OPTIONS request to an object

No

-

APIs for Object Operations

Table 3 APIs for object operations

API

Compatible with Parallel File Systems

Differences

Uploading an object with PUT

Yes
  • Headers not supported: x-obs-storage-class, x-obs-website-redirect-location, success-action-redirect, and x-obs-expires
  • Objects uploaded using this API cannot be directly stored in the Infrequent Access or Archive storage class and are stored in the Standard storage class by default. You can later change the storage class by using a lifecycle rule or modifying the metadata.
  • Directories that do not exist in the request path can be automatically created. For example, if you perform the following PUT operation and there is the dir01 directory in the parallel file system but there are no dir02 and dir03 directories, the dir02 and dir03 directories are automatically created. That is, after the command is executed successfully, dir02 and dir03 directories and file01 object are created. 
    PUT /dir01/dir02/dir03/file01 HTTP/1.1

Uploading an object with POST

Yes

Headers not supported: x-obs-storage-class, x-obs-website-redirect-location, success-action-redirect, and x-obs-expires

Copying an object

Yes

Data can be replicated only between parallel file systems or buckets that are in the same cluster.

Downloading an object

Yes

-

Obtaining object metadata

Yes

-

Deleting an object

Yes

-

Batch deleting objects

Yes

-

Restoring Archive objects

Yes

-

Appending data to an object

No

Parallel file systems do not support this API. You can obtain the file length by calling the API for obtaining object metadata and then call the API for modifying an object to append data.

Configuring object ACL

Yes

-

Obtaining an object ACL

Yes

-

Modifying object metadata

Yes

In a parallel file system, the storage class of a directory cannot be changed. To change the storage class of a file in the directory, modify the metadata of the file or use a lifecycle rule to change the storage class of files in batches.

Modifying an object

Yes

This API is only for parallel file systems and is not supported for OBS buckets.

This API cannot work with Infrequent Access or Archive files in parallel file systems.

Truncating an object

Yes

This API is only for parallel file systems and is not supported for OBS buckets.

This API cannot work with Infrequent Access or Archive files in parallel file systems.

Renaming an object

Yes

This API is only for parallel file systems and is not supported for OBS buckets.

Configuring object tags

No

-

Obtaining object tags

No

-

Deleting object tags

No

-

Configuring WORM retention for an object

No

-

APIs for Multipart Uploads

Table 4 APIs for multipart uploads

API

Compatible with Parallel File Systems

Differences

Listing the initiated multipart uploads in a bucket

Yes

-

Initiating a multipart upload

Yes

-

Uploading parts

Yes

-

Copying parts

Yes

Copying parts is not supported for an appended file.

Listing uploaded parts

Yes

-

Assembling parts

Yes

-

Aborting a multipart upload

Yes

-

Permissions Configuration

The use cases and main functions of object access control also work on files in parallel file systems. For more information, see Permissions Configuration Guide.

Differences Between File and Object Permission Configurations

To exactly match a specific directory, the resource path in the policy must end with a slash (/). When checking permissions, parallel file systems consider objects as directories. Since objects do not end with a slash (/), a parallel file system will add a slash (/) to the end of objects and then perform a policy matching.

IAM Permission Configuration Examples

Example 1: Grant a user the permissions required to download dir_1, excluding its subdirectories.

In the following configuration, the resource path ends with a slash (/). In such case, a success response can be returned when dir_1 or dir_1/ is contained in the URL of a head request.

Note that this configuration is not applied to subdirectories or files in dir_1. Therefore, a failure response will be returned if a head request is sent to dir_1/file1.

{
    "Version": "1.1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "obs:object:GetObject",
            ],
            "Resource": [
                "obs:*:*:object:examplebucket/dir_1/",
            ]
        }
    ]
}

Example 2: Grant a user the permissions required to download dir_1 and its subdirectories.

In the following configuration, the resource path uses prefix-based matching and ends with a wildcard (*). In such case, a success response can be returned when a head request is sent to dir_1/file1.

{
    "Version": "1.1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "obs:object:GetObject",
            ],
            "Resource": [
                "obs:*:*:object:examplebucket/dir_1/*",
            ]
        }
    ]
}

Policy Configuration Examples

Example: Grant a user the permissions required to download dir_1, excluding its subdirectories.

In the following configuration, the resource path ends with a slash (/). In such case, a success response can be returned when dir_1 or dir_1/ is contained in the URL of a head request.

{
    "Statement":[
    {
      "Sid":"test",
      "Effect":"Allow",
      "Principal": {"ID": ["domain/b4bf1b36d9ca43d984fbcb9491b6fce9:user/71f3901173514e6988115ea2c26d1999"]},
      "Action":["*"],
      "Resource":[
        "examplebucket/dir_1/",
      ]
    }
  ]
}

Lifecycle Management

The use cases and main functions of object lifecycle management also work on files in parallel file systems. For more information, see Creating a Lifecycle Rule.

To manage the lifecycle using SDKs, see SDK Overview.

Differences Between File and Object Lifecycle Management

  • You can configure lifecycle rules to manage files and directories. Directories cannot be transitioned to the Archive storage class, but empty directories can be deleted upon expiration.
  • Parallel file systems do not support versioning. Therefore, you cannot configure deletion upon expiration or transition between storage classes for historical versions of objects.
  • Each parallel file system can have a maximum of 20 lifecycle rules.
  • The time applied for a lifecycle rule to work on a file is when the data of the file was last updated.
  • Lifecycle rules cannot transition files to the Deep Archive storage.
  • A lifecycle rule configured for a parallel file system has limits on how many directories it can apply to. It will take longer time to execute the actions defined in the rule if:
    1. There are over 100,000 level-1 subdirectories in each directory.
    2. There are over 10 million subdirectories (folders) matching the specified prefix in total.
    3. There are over 30 million files matching the specified prefix in total.

Notes for Lifecycle Management

  • If renamed files or files in a renamed directory meet the conditions specified in a lifecycle rule, the rule applies based on the most recent data update, not the renaming time. In addition, the effective time of the lifecycle rule may be delayed for up to seven days.
  • For a file copy on a client, the lifecycle rule determines when to expire the file copy or transition it to the Archive storage class based on the file copy creation time.
    • For example, if a file, src.txt, was created on January 1, 2019, and was then copied to the des.txt file by running the cp -a src.txt des.txt command on September 1, 2019. Then the lifecycle rule calculated when to perform the specified actions on the file copy based on September 1, 2019.
  • A lifecycle rule scans directories periodically, starting from the deepest directory, and deletes those meeting expiration conditions. Empty directories that meet the conditions will be deleted, while non-empty ones will be ignored. Scan intervals (usually seven days) vary depending on cluster configurations. For this mechanism, a single-level directory that meets the expiration conditions will be deleted 0 to 7 days after it has been emptied. Accordingly, a two-level directory will be deleted 0 to 14 days after it has been emptied. Each time a directory level is added, the waiting time for deletion increases by seven days.
  • If a file or directory is mistakenly stored under a path whose prefix matches the filter in a lifecycle rule, it must be removed at least 24 hours before the rule is scheduled to apply. If you remove the file or directory less than 24 hours before it will be processed by the lifecycle rule, the file or directory may still be processed.
  • If the last-level directory in the prefix filter ends with a slash (/), the directory itself will not be deleted by the lifecycle rule. If the last-level directory does not end with a slash (/) and meets the lifecycle conditions, the directory will be deleted.
Parent Topic: Parallel File System