Using OBS Parallel File Systems

Parallel File System (PFS), a sub-product of OBS, is a high-performance file system, with only milliseconds of access latency. PFS can support terabytes of bandwidth and can handle millions of IOPS, which makes it ideal for processing high-performance computing (HPC) workloads. PFS outperforms OBS buckets. For details, see About Parallel File System.

In commercial deployments, you are advised to use parallel file systems instead of OBS buckets.

Technical Description

CCE allows you to mount OBS parallel file systems by using obsfs. For details about obsfs, see Introduction to obsfs.

An obsfs resident process is generated each time an object storage volume generated from the parallel file system is attached. The following shows an example.

Click to enlarge

Recommended Usage

You are advised to reserve 1 GiB memory for each obsfs process. For example, for a node with 4 vCPUs and 8 GiB memory, the obsfs parallel file systems should be mounted to no more than eight pods.

obsfs resident processes run on the node. If the consumed memory exceeds the upper limit of the node, the node becomes abnormal.

On a node with 4 vCPUs and 8 GiB memory, if more than 100 pods are mounted with parallel file systems, the node will be unavailable.

You are advised to control the number of pods mounted with parallel file systems on a single node.

Performance Metrics

Table 1 lists the performance test result of obsfs resident processes (for reference only).

**Table 1** obsfs resource consumption
Test Item	Memory Usage
Long-term stable running	About 50 MiB
Concurrently writing files of 10 MiB (two processes)	About 110 MiB
Concurrently writing files of 10 MiB (four processes)	About 220 MiB
Writing files of 100 GiB (single process)	About 300 MiB