Help Center> Cloud Container Engine> User Guide> Storage> Overview
Updated on 2024-03-11 GMT+08:00
View PDF

Overview

Container Storage

CCE container storage is implemented based on Kubernetes container storage APIs (CSI). CCE integrates multiple types of cloud storage and covers different application scenarios. CCE is fully compatible with Kubernetes native storage services, such as emptyDir, hostPath, secret, and ConfigMap.

Figure 1 Container storage types

CCE allows workload pods to use multiple types of storage:

  • In terms of implementation, storage supports Container Storage Interface (CSI) and Kubernetes native storage.

    Type

    Description

    CSI

    An out-of-tree volume add-on, which specifies the standard container storage API and allows storage vendors to use standard custom storage plugins that are mounted using PVCs and PVs without the need to add their plugin source code to the Kubernetes repository for unified build, compilation, and release. CSI is a recommended in Kubernetes 1.13 and later versions.

    Kubernetes native storage

    An "in-tree" volume add-on that is built, compiled, and released with the Kubernetes repository.
  • In terms of storage media, storage can be classified as cloud storage, local storage, and Kubernetes resource objects.

    Type

    Description

    Application Scenario

    Cloud storage

    The storage media is provided by storage vendors. Storage volumes of this type are mounted using PVCs and PVs.

    Data requires high availability or needs to be shared, for example, logs and media resources.

    Select a proper cloud storage type based on the application scenario. For details, see Cloud Storage Comparison.

    Local storage

    The storage media is the local data disk or memory of the node. The local persistent volume is a customized storage type provided by CCE and mounted using PVCs and PVs through the CSI. Other storage types are Kubernetes native storage.

    Non-HA data requires high I/O and low latency.

    Select a proper local storage type based on the application scenario. For details, see Local Storage Comparison.

    Kubernetes resource objects

    ConfigMaps and secrets are resources created in clusters. They are special storage types and are provided by tmpfs (RAM-based file system) on the Kubernetes API server.

    ConfigMaps are used to inject configuration data to pods.

    Secrets are used to transmit sensitive information such as passwords to pods.

Cloud Storage Comparison

Item

EVS

SFS

SFS Turbo

OBS

Definition

EVS offers scalable block storage for cloud servers. With high reliability, high performance, and rich specifications, EVS disks can be used for distributed file systems, dev/test environments, data warehouses, and high-performance computing (HPC) applications.

Expandable to petabytes, SFS provides fully hosted shared file storage, highly available and stable to handle data- and bandwidth-intensive applications in HPC, media processing, file sharing, content management, and web services.

Expandable to 320 TB, SFS Turbo provides fully hosted shared file storage, which is highly available and stable, to support small files and applications requiring low latency and high IOPS. You can use SFS Turbo in high-traffic websites, log storage, compression/decompression, DevOps, enterprise OA, and containerized applications.

Object Storage Service (OBS) provides massive, secure, and cost-effective data storage for you to store data of any type and size. You can use it in enterprise backup/archiving, video on demand (VoD), video surveillance, and many other scenarios.

Data storage logic

Stores binary data and cannot directly store files. To store files, format the file system first.

Stores files and sorts and displays data in the hierarchy of files and folders.

Stores files and sorts and displays data in the hierarchy of files and folders.

Stores objects. Files directly stored automatically generate the system metadata, which can also be customized by users.

Access mode

Accessible only after being mounted to ECSs or BMSs and initialized.

Mounted to ECSs or BMSs using network protocols. A network address must be specified or mapped to a local directory for access.

Supports the Network File System (NFS) protocol (NFSv3 only). You can seamlessly integrate existing applications and tools with SFS Turbo.

Accessible through the Internet or Direct Connect (DC). Specify the bucket address and use transmission protocols such as HTTP or HTTPS.

Static storage volumes

Supported. For details, see Using an Existing EVS Disk Through a Static PV.

Supported. For details, see Using an Existing SFS File System Through a Static PV.

Supported. For details, see Using an Existing SFS Turbo File System Through a Static PV.

Supported. For details, see Using an Existing OBS Bucket Through a Static PV.

Dynamic storage volumes

Supported. For details, see Using an EVS Disk Through a Dynamic PV.

Supported. For details, see Using an SFS File System Through a Dynamic PV.

Not supported

Supported. For details, see Using an OBS Bucket Through a Dynamic PV.

Features

Non-shared storage. Each volume can be mounted to only one node.

Shared storage featuring high performance and throughput

Shared storage featuring high performance and bandwidth

Shared, user-mode file system

Application scenarios

HPC, enterprise core cluster applications, enterprise application systems, and dev/test

NOTE:

HPC apps here require high-speed and high-IOPS storage, such as industrial design and energy exploration.

HPC, media processing, content management, web services, big data, and analysis applications

NOTE:

HPC apps here require high bandwidth and shared file storage, such as gene sequencing and image rendering.

High-traffic websites, log storage, DevOps, and enterprise OA

Big data analytics, static website hosting, online video on demand (VoD), gene sequencing, intelligent video surveillance, backup and archiving, and enterprise cloud boxes (web disks)

Capacity

TB

SFS 1.0: PB

General-purpose: TB

EB

Latency

1–2 ms

SFS 1.0: 3–20 ms

General-purpose: 1–5 ms

10 ms

IOPS/TPS

33,000 for a single disk

SFS 1.0: 2000

General-purpose: up to 100,000

Tens of millions

Bandwidth

MB/s

SFS 1.0: GB/s

General-purpose: up to GB/s

TB/s

Local Storage Comparison

Item

Local PV

Local Ephemeral Volume

emptyDir

hostPath

Definition

Node's local disks form a storage pool (VolumeGroup) through LVM. LVM divides them into logical volumes (LVs) and mounts them to pods.

Kubernetes native emptyDir, where node's local disks form a storage pool (VolumeGroup) through LVM. LVs are created as the storage media of emptyDir and mounted to pods. LVs deliver better performance than the default storage medium of emptyDir.

Kubernetes native emptyDir. Its lifecycle is the same as that of a pod. Memory can be specified as the storage media. When the pod is deleted, the emptyDir volume is deleted and its data is lost.

Used to mount a file directory of the host where a pod is located to a specified mount point of the pod.

Features

Low-latency, high-I/O, and non-HA persistent volume.

Storage volumes are non-shared storage and bound to nodes through labels. Therefore, storage volumes can be mounted only to a single pod.

Local temporary volume. The storage space is from local LVs.

Local temporary volume. The storage space comes from the local kubelet root directory or memory.

Used to mount files or directories of the host file system. Host directories can be automatically created. Pods can be migrated (not bound to nodes).

Storage volume mounting

Static storage volumes are not supported.

Using a Local PV Through a Dynamic PV is supported.

For details, see Using a Local EV.

For details, see Using a Temporary Path.

For details, see hostPath.

Application scenarios

High I/O requirements and built-in HA solutions of applications, for example, deploying MySQL in HA mode.
  • Scratch space, such as for a disk-based merge sort
  • Checkpointing a long computation for recovery from crashes
  • Saving the files obtained by the content manager container when web server container data is used
  • Scratch space, such as for a disk-based merge sort
  • Checkpointing a long computation for recovery from crashes
  • Saving the files obtained by the content manager container when web server container data is used

Requiring a node file, for example, if Docker is used, you can use hostPath to mount the /var/lib/docker path of the node.

NOTICE:

Avoid using hostPath volumes as much as possible, as they are prone to security risks. If hostPath volumes must be used, they can only be applied to files or directories and mounted in read-only mode.

Enterprise Project Support

To use this function, the Everest add-on must be upgraded to v1.2.33 or later.

  • Automatically creating storage:

    CCE allows you to specify an enterprise project when creating EVS disks and OBS PVCs. The created storage resources (EVS disks and OBS) belong to the specified enterprise project. The enterprise project can be the enterprise project to which the cluster belongs or the default enterprise project.

    If no enterprise project is specified, the enterprise project specified in StorageClass will be used by default for creating storage resources.
    • For a custom StorageClass, you can specify an enterprise project in StorageClass. For details, see Specifying an Enterprise Project for Storage Classes. If no enterprise project is specified in StorageClass, the default enterprise project is used.
    • For the csi-disk and csi-obs storage classes provided by CCE, the created storage resources belong to the default enterprise project.
  • Use existing storage:

    When you create a PVC using a PV, ensure that everest.io/enterprise-project-id specified in the PVC and PV are the same because an enterprise project has been specified during storage resource creation. Otherwise, the PVC and PV cannot be bound.

Documentation

Parent topic: Storage