Why Service Performance Does Not Meet the Expectation?

The underlying resources of CCI are shared by multiple tenants. To ensure service stability, the underlying layer of CCI has traffic control on disk I/O. For containers, the read and write operations on the rootfs directory and the standard output of workload logs are limited. If the service performance does not meet your expectations, locate the fault based on the following causes:

Cause 1: Service containers print a large number of log files to stdout streams.

The underlying layer of CCI limits the forwarding traffic of standard output. If the service log traffic is greater than 1 MB/s, you are advised to use log volumes to report logs to Application Operations Management (AOM). For details, see Log Management. Alternatively, you can output logs to FlexVolume or persistent EVS or SFS volumes, and use a sidecar to run an open-source component such as Fluent Bit to report logs to your self-built log centers. If a large number of logs are printed to stdout streams, service performance may be affected due to the forwarding traffic limits.

Cause 2: Service containers have high I/O read and write operations on the rootfs disk.

CCI limits the I/O traffic of the container system disk (rootfs). If a process may have high disk I/O operations (bandwidth > 6 MB/s, IOPS > 1,000) during the service running or is sensitive to the disk I/O performance, do not execute it in rootfs. For example, frequently printing logs to rootfs may cause frequent read and write operations on it. You can place service-related configuration files or files that are not frequently read and written in rootfs. For high I/O file operations, select log volumes or FlexVolume (created or deleted with the pod), or persistent EVS or SFS volumes based on your service requirements. Executing high I/O operations in the rootfs disk may affect service performance due to disk traffic limiting.

If the preceding problems occur, adjust configurations based on the description to prevent service performance from being affected.