Updated on 2024-03-04 GMT+08:00

Lifecycle Management

The application scenarios and main functions of lifecycle management for PFS is consistent with those of OBS. For more information, see Lifecycle Management.

For details about the SDK reference for lifecycle management, see SDK Overview.

Differences from the Object Lifecycle Management

  • You can use the lifecycle management function to manage files. Lifecycle management cannot transition a folder to the Archive storage class, but it can delete an empty folder upon expiration.
  • You can use APIs to configure lifecycle rules to expire and then delete the specified files or to transition the specified files to the Archive or Infrequent Access storage class. Currently, OBS Console allows only deletion upon expiration and transition to the Archive storage class. PFS does not support versioning, so lifecycle rules (including deletion upon expiration, and transition to the Archive or Infrequent Access storage class) that are related to versioning cannot be applied.
  • If direct reading is enabled for a parallel file system, you can read files stored in the Archive storage class without restoring them first.
  • A maximum of 20 lifecycle rules can be configured for a parallel file system.
  • A lifecycle rule adjusts its baseline time to keep in line with when relevant files have been most recently updated.
  • Lifecycle rules cannot transition files to the Deep Archive storage.
  • After a lifecycle rule is configured for a parallel file system, there are limits on how many directories that the rule can be applied to. If the configuration exceeds the limit, the lifecycle rule execution will be prolonged.
    1. There can be no more than 100,000 level-1 subdirectories in each directory.
    2. There can be no more than 10 million subdirectories (folders) matching the prefix defined in the rule in total.
    3. The total number of files matching the prefix defined in the rule cannot exceed 30 million.

Other Descriptions

  • If you rename a file or a folder, and the renamed file or files in the renamed folder meet the requirements of a lifecycle rule, the baseline time of the lifecycle rule will be the time when the content data of the files was most recently updated, not when the file was renamed. In addition, the effective time of the lifecycle rule may be delayed for up to 7 days.
  • For a file copy on a client, its expiration time or time when it is transitioned to the Archive storage class is based on when the file was most recently copied.
    • For example, if a file, src.txt, was created on January 1, 2019, and was then copied to the des.txt file by running the cp -a src.txt des.txt command on September 1, 2019. Then the lifecycle rule is based on September 1, 2019.
  • In the lifecycle rule of a parallel file system, directories in the file system are periodically scanned, and then deleted if they meet the expiration conditions. Scan intervals (usually seven days) vary depending on cluster configurations. A scan starts from the deepest directory, and the scanned empty directories that meet the expiration conditions will be deleted, but those non-empty directories will not be processed. For this mechanism, a single-level directory that meets the expiration conditions will be deleted 0 to 7 days after it has been emptied. Accordingly, a two-level directory will be deleted 0 to 14 days after it has been emptied. Each time a directory level is added, the waiting time for deletion increases by seven days.