Dynamically Mounting an EVS Volume to a StatefulSet
Scenario
Dynamic mounting is available only for creating a StatefulSet. It is implemented through a volume claim template (the volumeClaimTemplates field) and depends on dynamic creation of PVs through StorageClass. In this mode, each pod in a multi-pod StatefulSet is associated with a unique PVC and PV. After a pod is rescheduled, the original PV can still be mounted to it based on the PVC name. In the common mounting mode for a Deployment, if ReadWriteMany is supported, multiple pods of the Deployment will be mounted to the same underlying storage.
When updating a StatefulSet in Kubernetes, it is not allowed to add or delete the volumeClaimTemplates field. This field can only be configured during the creation of the StatefulSet.
Prerequisites
- You have created a cluster.
- If you want to create a cluster by running commands, kubectl has been used to connect to the cluster. For details, see Connecting to a Cluster Using kubectl.
Constraints
- Not all regions support EVS volumes. View the regions where EVS volumes are supported on the console. You can also view Function Overview to learn about all regions where EVS volumes are supported.
- The cluster version must be v1.27.8-r0, v1.28.6-r0, or later. If the cluster version does not meet the requirements, you need to upgrade the cluster.
- No more than 10 EVS disks can be attached to each workload in a cluster. If more than 10 EVS disks are attached, the workload may run abnormally.
Dynamically Creating and Mounting an EVS Volume
By specifying the StorageClass, you can create a unique PVC and PV for each pod of a StatefulSet on the console or using kubectl to ensure data durability after rescheduling.
Dynamically mount and use storage volumes. For details about other parameters, see Creating a StatefulSet.
- Log in to the CCE console and click the cluster name to access the cluster console.
- In the navigation pane on the left, choose Workloads. Then click the StatefulSets tab.
- Click Create Workload in the upper right corner. On the displayed page, click Data Storage in the Container Settings area and click Add Volume to select VolumeClaimTemplate.
- In the upper right corner, click Create PVC. In the displayed dialog box, configure the parameters.
Click Create.Figure 1 Creating a PVC
Parameter
Description
PVC Type
In this example, select EVS.
PVC Name
Enter the name of the PVC. After a PVC is created, a suffix is automatically added based on the number of pods. The format is <Custom PVC name>-<Serial number>, for example, example-0.
Namespace
A namespace is a conceptual grouping of resources or objects. Each namespace provides isolation for data from other namespaces.
After a cluster is created, the default namespace is created by default. If there is no special requirement, select the default namespace.
Creation Method
You can select Dynamically provision to create a PVC, PV, and underlying storage on the console in cascading mode.
Storage Classes
The StorageClass for EVS volumes is csi-disk.
(Optional) PV Name Prefix
This parameter specifies the name of the underlying storage that is automatically created. The actual underlying storage name is in the format of "PV name prefix + PVC UID". If this parameter is left blank, the default prefix pvc will be used.
For example, if the storage volume name prefix is set to test, the actual underlying storage name is test-{UID}.
AZ
Select the AZ of the EVS disk. After the EVS disk is created, the AZ cannot be changed.
In this example, select AZ3.
Disk Type
Select an EVS disk type. EVS disk types vary depending on regions. Obtain the available EVS types on the console.
In this example, select High I/O.
NOTE:
General-purpose SSD V2 disks allow you to specify the disk IOPS and throughput. For details, see EVS performance data.
Capacity (GiB)
Capacity of the requested storage volume.
Set it to 10 in this example.
Access Mode
EVS volumes support only ReadWriteOnce, indicating that a storage volume can only be mounted to one pod in read/write mode. For details, see Volume Access Modes.
Encryption
Configure whether to encrypt underlying storage. If you select Enabled (key), an encryption key must be configured. Currently, encryption is not supported.
Enterprise Project
The default enterprise project, the enterprise project to which the cluster belongs, or the enterprise project specified by StorageClass is available.
Resource Tag
You can add resource tags to classify resources.
You can create predefined tags on the TMS console. The predefined tags are available to all resources that support tags. You can use predefined tags to improve the tag creation and resource migration efficiency. For details, see Creating Predefined Tags.
CCE automatically creates system tags CCE-Cluster-ID={Cluster ID}, CCE-Cluster-Name={Cluster name}, and CCE-Namespace={Namespace name}. These tags cannot be modified.
NOTE:
After a dynamic PV of the EVS type is created, the resource tags cannot be updated on the CCE console. To update these resource tags, go to the EVS console.
- Enter the path to which the volume is mounted.
In this example, the volume is mounted to the /data path of the container. The container data generated in this path is stored in the EVS disk.
Figure 2 Parameters for mounting a storage volumeTable 1 Parameters for mounting a storage volume Parameter
Description
Mount Path
Enter a mount path, for example, /data.
This parameter indicates the container path that the volume will be mounted to. Do not mount the volume to a system directory such as / or /var/run. This may cause container errors. Mount the volume to an empty directory. If the directory is not empty, ensure that there are no files that affect container startup. If there are such files, they will be replaced, which will lead to a container startup or workload creation failure.NOTICE:If a volume is mounted to a high-risk directory, use an account with minimum permissions to start the container, or high-risk files on the host may be damaged.
Subpath
Enter the subpath of the storage volume and mount a path in the storage volume to the container. In this way, different folders of the same storage volume can be used in a single pod. data, for example, indicates that data in the mount path of the container is stored in the data directory of the storage volume. If this parameter is left blank, the root path is used by default.
Permission
- Read-only: You can only read the data in the mounted volume.
- Read-write: You can modify the volume mounted to the path. Newly written data will not be migrated if the container is migrated, which may cause data loss.
- Configure other parameters and click Create Workload. After the workload is created, the data in the container mount directory will be persistently stored. Verify the storage by referring to Verifying Data Persistence.
- Use kubectl to connect to the cluster.
- Create a file named statefulset-evs.yaml. In this example, the EVS volume is mounted to the /data path.
apiVersion: apps/v1 kind: StatefulSet metadata: name: statefulset-evs namespace: default spec: selector: matchLabels: app: statefulset-evs template: metadata: labels: app: statefulset-evs spec: containers: - name: container-1 image: nginx:latest volumeMounts: - name: pvc-disk # The value must be the same as that in the volumeClaimTemplates field. mountPath: /data # Location where the storage volume is mounted imagePullSecrets: - name: default-secret serviceName: statefulset-evs # Headless Service name replicas: 2 volumeClaimTemplates: - apiVersion: v1 kind: PersistentVolumeClaim metadata: name: pvc-disk namespace: default annotations: everest.io/disk-volume-type: SAS # EVS disk type everest.io/enterprise-project-id: <your_project_id> # (Optional) Enterprise project ID. If an enterprise project is specified, use the same enterprise project when creating a PVC. Otherwise, the PVC cannot be bound to a PV. everest.io/disk-iops: '3000' # (Optional) IOPS of only a GPSSD2 EVS disk everest.io/disk-throughput: '125' # (Optional) Throughput of only a GPSSD2 EVS disk everest.io/csi.volume-name-prefix: test # (Optional) Storage volume name prefix of the automatically created underlying storage labels: failure-domain.beta.kubernetes.io/region: <your_region> # Replace the region with the one where the cluster is located. failure-domain.beta.kubernetes.io/zone: <your_zone> # Replace the AZ with the one where the EVS disk is located. spec: accessModes: - ReadWriteOnce # The value must be ReadWriteOnce for EVS disks. resources: requests: storage: 10Gi # EVS disk capacity, in GiB. The value ranges from 1 to 32768. storageClassName: csi-disk # The StorageClass of the EVS disk --- apiVersion: v1 kind: Service metadata: name: statefulset-evs # Headless Service name namespace: default labels: app: statefulset-evs spec: selector: app: statefulset-evs clusterIP: None ports: - name: statefulset-evs targetPort: 80 nodePort: 0 port: 80 protocol: TCP type: ClusterIP
Table 2 Key parameters Parameter
Mandatory
Description
failure-domain.beta.kubernetes.io/region
Yes
Region of the cluster.
For example, cn-east-3. For details, see Regions and Endpoints.
failure-domain.beta.kubernetes.io/zone
Yes
AZ of the EVS disk.
You can obtain all supported AZs by querying the AZ list.
everest.io/disk-volume-type
Yes
EVS disk type. All letters are in uppercase. EVS disk types vary depending on regions. Obtain the available EVS types on the console.- SAS: high I/O
- SSD: ultra-high I/O
- GPSSD: general-purpose SSD
- ESSD: extreme SSD
- GPSSD2: general-purpose SSD v2. You need to specify the everest.io/disk-iops and everest.io/disk-throughput annotations.
everest.io/disk-iops
No
Preconfigured IOPS, which is supported only by general-purpose SSD v2 EVS disks.- The IOPS of general-purpose SSD v2 EVS disks ranges from 3000 to 128000, and the maximum value is 500 times of the capacity (GiB).
If the IOPS of general-purpose SSD v2 disks is greater than 3000, extra IOPS will be billed. For details, see Price Calculator.
everest.io/disk-throughput
No
Preconfigured throughput, which is supported only by general-purpose SSD v2 EVS disks.
The value ranges from 125 MiB/s to 1,000 MiB/s. The maximum value is a quarter of IOPS.
If the throughput is greater than 125 MiB/s, extra throughput will be billed. For details, see Price Calculator.
everest.io/enterprise-project-id
No
Enterprise project ID of the EVS disk. This parameter is optional. If an enterprise project is specified, use the same enterprise project when creating a PVC, or the PVC cannot be bound to a PV.
To obtain an enterprise project ID, log in to the EPS console, click the name of the target enterprise project, and copy the enterprise project ID.
everest.io/disk-volume-tags
No
You can add resource tags to classify resources. This field is optional.
You can create predefined tags on the TMS console. The predefined tags are available to all resources that support tags. You can use predefined tags to improve the tag creation and resource migration efficiency. For details, see Creating Predefined Tags.
CCE automatically creates system tags CCE-Cluster-ID={Cluster ID}, CCE-Cluster-Name={Cluster name}, and CCE-Namespace={Namespace name}. These tags cannot be modified.
everest.io/csi.volume-name-prefix
No
This parameter is optional. It specifies the name of the underlying storage that is automatically created. The actual underlying storage name is in the format of "PV name prefix + PVC UID". If this parameter is left blank, the default prefix pvc will be used.
Enter 1 to 26 characters that cannot start or end with a hyphen (-). Only lowercase letters, digits, and hyphens (-) are allowed.
For example, if the storage volume name prefix is set to test, the actual underlying storage name is test-{UID}.
storage
Yes
Requested PVC capacity, in Gi. The value ranges from 1 to 32768.
storageClassName
Yes
The StorageClass for EVS volumes is csi-disk.
- Run the following command to create a workload that the EVS volume is mounted to:
kubectl apply -f statefulset-evs.yaml
After the workload is created, the data in the container mount directory will be persistently stored. Verify the storage by referring to Verifying Data Persistence.
Dynamically mount and use storage volumes. For details about other parameters, see Creating a StatefulSet.
- Log in to the CCE console and click the cluster name to access the cluster console.
- In the navigation pane on the left, choose Workloads. Then click the StatefulSets tab.
- Click Create Workload in the upper right corner. On the displayed page, click Data Storage in the Container Settings area and click Add Volume to select VolumeClaimTemplate.
- In the upper right corner, click Create PVC. In the displayed dialog box, configure the parameters.
Click Create.Figure 1 Creating a PVC
Parameter
Description
PVC Type
In this example, select EVS.
PVC Name
Enter the name of the PVC. After a PVC is created, a suffix is automatically added based on the number of pods. The format is <Custom PVC name>-<Serial number>, for example, example-0.
Namespace
A namespace is a conceptual grouping of resources or objects. Each namespace provides isolation for data from other namespaces.
After a cluster is created, the default namespace is created by default. If there is no special requirement, select the default namespace.
Creation Method
You can select Dynamically provision to create a PVC, PV, and underlying storage on the console in cascading mode.
Storage Classes
The StorageClass for EVS volumes is csi-disk.
(Optional) PV Name Prefix
This parameter specifies the name of the underlying storage that is automatically created. The actual underlying storage name is in the format of "PV name prefix + PVC UID". If this parameter is left blank, the default prefix pvc will be used.
For example, if the storage volume name prefix is set to test, the actual underlying storage name is test-{UID}.
AZ
Select the AZ of the EVS disk. After the EVS disk is created, the AZ cannot be changed.
In this example, select AZ3.
Disk Type
Select an EVS disk type. EVS disk types vary depending on regions. Obtain the available EVS types on the console.
In this example, select High I/O.
NOTE:
General-purpose SSD V2 disks allow you to specify the disk IOPS and throughput. For details, see EVS performance data.
Capacity (GiB)
Capacity of the requested storage volume.
Set it to 10 in this example.
Access Mode
EVS volumes support only ReadWriteOnce, indicating that a storage volume can only be mounted to one pod in read/write mode. For details, see Volume Access Modes.
Encryption
Configure whether to encrypt underlying storage. If you select Enabled (key), an encryption key must be configured. Currently, encryption is not supported.
Enterprise Project
The default enterprise project, the enterprise project to which the cluster belongs, or the enterprise project specified by StorageClass is available.
Resource Tag
You can add resource tags to classify resources.
You can create predefined tags on the TMS console. The predefined tags are available to all resources that support tags. You can use predefined tags to improve the tag creation and resource migration efficiency. For details, see Creating Predefined Tags.
CCE automatically creates system tags CCE-Cluster-ID={Cluster ID}, CCE-Cluster-Name={Cluster name}, and CCE-Namespace={Namespace name}. These tags cannot be modified.
NOTE:
After a dynamic PV of the EVS type is created, the resource tags cannot be updated on the CCE console. To update these resource tags, go to the EVS console.
- Enter the path to which the volume is mounted.
In this example, the volume is mounted to the /data path of the container. The container data generated in this path is stored in the EVS disk.
Figure 2 Parameters for mounting a storage volumeTable 1 Parameters for mounting a storage volume Parameter
Description
Mount Path
Enter a mount path, for example, /data.
This parameter indicates the container path that the volume will be mounted to. Do not mount the volume to a system directory such as / or /var/run. This may cause container errors. Mount the volume to an empty directory. If the directory is not empty, ensure that there are no files that affect container startup. If there are such files, they will be replaced, which will lead to a container startup or workload creation failure.NOTICE:If a volume is mounted to a high-risk directory, use an account with minimum permissions to start the container, or high-risk files on the host may be damaged.
Subpath
Enter the subpath of the storage volume and mount a path in the storage volume to the container. In this way, different folders of the same storage volume can be used in a single pod. data, for example, indicates that data in the mount path of the container is stored in the data directory of the storage volume. If this parameter is left blank, the root path is used by default.
Permission
- Read-only: You can only read the data in the mounted volume.
- Read-write: You can modify the volume mounted to the path. Newly written data will not be migrated if the container is migrated, which may cause data loss.
- Configure other parameters and click Create Workload. After the workload is created, the data in the container mount directory will be persistently stored. Verify the storage by referring to Verifying Data Persistence.
- Use kubectl to connect to the cluster.
- Create a file named statefulset-evs.yaml. In this example, the EVS volume is mounted to the /data path.
apiVersion: apps/v1 kind: StatefulSet metadata: name: statefulset-evs namespace: default spec: selector: matchLabels: app: statefulset-evs template: metadata: labels: app: statefulset-evs spec: containers: - name: container-1 image: nginx:latest volumeMounts: - name: pvc-disk # The value must be the same as that in the volumeClaimTemplates field. mountPath: /data # Location where the storage volume is mounted imagePullSecrets: - name: default-secret serviceName: statefulset-evs # Headless Service name replicas: 2 volumeClaimTemplates: - apiVersion: v1 kind: PersistentVolumeClaim metadata: name: pvc-disk namespace: default annotations: everest.io/disk-volume-type: SAS # EVS disk type everest.io/enterprise-project-id: <your_project_id> # (Optional) Enterprise project ID. If an enterprise project is specified, use the same enterprise project when creating a PVC. Otherwise, the PVC cannot be bound to a PV. everest.io/disk-iops: '3000' # (Optional) IOPS of only a GPSSD2 EVS disk everest.io/disk-throughput: '125' # (Optional) Throughput of only a GPSSD2 EVS disk everest.io/csi.volume-name-prefix: test # (Optional) Storage volume name prefix of the automatically created underlying storage labels: failure-domain.beta.kubernetes.io/region: <your_region> # Replace the region with the one where the cluster is located. failure-domain.beta.kubernetes.io/zone: <your_zone> # Replace the AZ with the one where the EVS disk is located. spec: accessModes: - ReadWriteOnce # The value must be ReadWriteOnce for EVS disks. resources: requests: storage: 10Gi # EVS disk capacity, in GiB. The value ranges from 1 to 32768. storageClassName: csi-disk # The StorageClass of the EVS disk --- apiVersion: v1 kind: Service metadata: name: statefulset-evs # Headless Service name namespace: default labels: app: statefulset-evs spec: selector: app: statefulset-evs clusterIP: None ports: - name: statefulset-evs targetPort: 80 nodePort: 0 port: 80 protocol: TCP type: ClusterIP
Table 2 Key parameters Parameter
Mandatory
Description
failure-domain.beta.kubernetes.io/region
Yes
Region of the cluster.
For example, cn-east-3. For details, see Regions and Endpoints.
failure-domain.beta.kubernetes.io/zone
Yes
AZ of the EVS disk.
You can obtain all supported AZs by querying the AZ list.
everest.io/disk-volume-type
Yes
EVS disk type. All letters are in uppercase. EVS disk types vary depending on regions. Obtain the available EVS types on the console.- SAS: high I/O
- SSD: ultra-high I/O
- GPSSD: general-purpose SSD
- ESSD: extreme SSD
- GPSSD2: general-purpose SSD v2. You need to specify the everest.io/disk-iops and everest.io/disk-throughput annotations.
everest.io/disk-iops
No
Preconfigured IOPS, which is supported only by general-purpose SSD v2 EVS disks.- The IOPS of general-purpose SSD v2 EVS disks ranges from 3000 to 128000, and the maximum value is 500 times of the capacity (GiB).
If the IOPS of general-purpose SSD v2 disks is greater than 3000, extra IOPS will be billed. For details, see Price Calculator.
everest.io/disk-throughput
No
Preconfigured throughput, which is supported only by general-purpose SSD v2 EVS disks.
The value ranges from 125 MiB/s to 1,000 MiB/s. The maximum value is a quarter of IOPS.
If the throughput is greater than 125 MiB/s, extra throughput will be billed. For details, see Price Calculator.
everest.io/enterprise-project-id
No
Enterprise project ID of the EVS disk. This parameter is optional. If an enterprise project is specified, use the same enterprise project when creating a PVC, or the PVC cannot be bound to a PV.
To obtain an enterprise project ID, log in to the EPS console, click the name of the target enterprise project, and copy the enterprise project ID.
everest.io/disk-volume-tags
No
You can add resource tags to classify resources. This field is optional.
You can create predefined tags on the TMS console. The predefined tags are available to all resources that support tags. You can use predefined tags to improve the tag creation and resource migration efficiency. For details, see Creating Predefined Tags.
CCE automatically creates system tags CCE-Cluster-ID={Cluster ID}, CCE-Cluster-Name={Cluster name}, and CCE-Namespace={Namespace name}. These tags cannot be modified.
everest.io/csi.volume-name-prefix
No
This parameter is optional. It specifies the name of the underlying storage that is automatically created. The actual underlying storage name is in the format of "PV name prefix + PVC UID". If this parameter is left blank, the default prefix pvc will be used.
Enter 1 to 26 characters that cannot start or end with a hyphen (-). Only lowercase letters, digits, and hyphens (-) are allowed.
For example, if the storage volume name prefix is set to test, the actual underlying storage name is test-{UID}.
storage
Yes
Requested PVC capacity, in Gi. The value ranges from 1 to 32768.
storageClassName
Yes
The StorageClass for EVS volumes is csi-disk.
- Run the following command to create a workload that the EVS volume is mounted to:
kubectl apply -f statefulset-evs.yaml
After the workload is created, the data in the container mount directory will be persistently stored. Verify the storage by referring to Verifying Data Persistence.
Verifying Data Persistence
- View the deployed application and files stored in the EVS volume.
- Run the following command to view the pod:
kubectl get pod | grep statefulset-evs
Expected output:statefulset-evs-0 1/1 Running 0 45s statefulset-evs-1 1/1 Running 0 28s
- Run the following command to verify that the EVS volume has been mounted to the /data path:
kubectl exec statefulset-evs-0 -- df | grep data
Expected output:
/dev/sdd 10255636 36888 10202364 0% /data
- Run the following command to view the created file in the /data path:
kubectl exec statefulset-evs-0 -- ls /data
Expected output:
lost+found
- Run the following command to view the pod:
- Run the following command to create a file named static in the /data path:
kubectl exec statefulset-evs-0 -- touch /data/static
- Run the following command to view the created file in the /data path:
kubectl exec statefulset-evs-0 -- ls /data
Expected output:
lost+found static
- Run the following command to delete the pod named web-evs-auto-0:
kubectl delete pod statefulset-evs-0
Expected output:
pod "statefulset-evs-0" deleted
- The StatefulSet controller automatically creates a replica with the same name as the pod. Run the following command to check whether the file in the /data path has been modified:
kubectl exec statefulset-evs-0 -- ls /data
Expected output:
lost+found static
The static file is retained, indicating that the data can be stored persistently.
Related Operations
Operation |
Description |
Procedure |
---|---|---|
Expanding the capacity of an EVS volume |
Quickly expand the capacity of a mounted EVS volume on the CCE console. |
|
Viewing events |
You can view event names, event types, number of occurrences, Kubernetes events, first occurrence time, and last occurrence time of the PVC or PV. |
|
Viewing a YAML file |
You can view, copy, and download the YAML files of a PVC or PV. |
|
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot