How Do I Expand the Disk Capacity of the CIA Add-on in an On-Premises Cluster?
Symptom
When the disk in the PVC storage on which the CIA add-on (kube-prometheus-stack) depends is full, the no space left on device error message is reported in the standard log output of prometheus-server-0 Pod. In this case, Prometheus cannot work properly. You need to expand the disk capacity of the node where the PVC is located and restart prometheus-server-0 Pod.
Procedure
- Check the PVC information of Prometheus and obtain the node and path where the PV bound to the PVC is located.
kubectl describe pvc pvc-prometheus-server-0 -nmonitoring|grep volume.kubernetes.io/selected-node
kubectl describe pvc pvc-prometheus-server-0 -nmonitoring|grep volume.kubernetes.io/targetPath
- Log in to the storage node, run the df -TH command to query the disk configuration of the node, and expand the capacity according to the disk configuration. After the capacity expansion is complete, you need to partition the disk. For details, see Extending Disk Partitions and File Systems (Linux) in the Elastic Volume Service User Guide.
- After the capacity expansion and partitioning are complete, restart Prometheus.
kubectl delete pod prometheus-server-0 -nmonitoring
If prometheus-server-0 is restarted, the CIA function of the container will be unavailable during the restart. Select an appropriate restart time.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot