StatefulSets

Overview of StatefulSet

All pods under a Deployment have the same characteristics except for the name and IP address. If required, a Deployment can use a pod template to create new pods. If not required, the Deployment can delete any one of the pods.

However, Deployments cannot meet the requirements in some distributed scenarios when each pod requires its own status or in a distributed database where each pod requires independent storage.

Distributed stateful applications involve different roles for different responsibilities. For example, databases work in active/standby mode, and pods depend on each other. To deploy stateful applications in Kubernetes, ensure pods meet the following requirements:

• Each pod must have a fixed identifier so that it can be recognized by other pods.
• Separate storage resources must be configured for each pod. In this way, the original data can be retrieved after a pod is deleted and restored. Otherwise, the pod status will be changed after the pod is rebuilt.

To address the preceding requirements, Kubernetes provides StatefulSets.

1. StatefulSets provide a fixed name for each pod following a fixed number ranging from 0 to N. After a pod is rescheduled, the pod name and the hostname remain unchanged.
2. StatefulSets use a headless Service to allocate a fixed domain name for each pod.
3. StatefulSets create PersistentVolumeClaims (PVCs) with fixed identifiers to ensure that pods can access the same persistent data after being rescheduled.

   

Creating a Headless Service

A headless Service is required by a StatefulSet for accessing pods.

Use the following file to describe the headless Service:

• spec.clusterIP: must be set to None to indicate a headless Service.
• spec.ports.port: number of the port for communication between pods.
• spec.ports.name: name of the port for communication between pods.

apiVersion: v1
kind: Service       # The object type is Service.
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
    - name: nginx     # Name of the port for communication between pods
      port: 80        # Number of the port for communication between pods
  selector:
    app: nginx        # Select the pod labeled with app:nginx.
  clusterIP: None     # Set this parameter to None, indicating a headless Service.

Run the following command to create a headless Service:

# kubectl create -f headless.yaml 
service/nginx created

After the Service is created, check the Service information.

# kubectl get svc
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
nginx        ClusterIP   None         <none>        80/TCP    5s

Creating a StatefulSet

The YAML definition of StatefulSets is basically the same as that of other objects. The differences are as follows:

• serviceName specifies the headless Service used by a StatefulSet. You are required to configure this parameter.
• volumeClaimTemplates is used to apply for a PVC. A template named data is defined, which will create a PVC for each pod. storageClassName specifies the persistent StorageClass. For details, see PersistentVolumes, PersistentVolumeClaims, and StorageClasses. volumeMounts specifies storage to mount to pods. If no storage is required, you can delete the volumeClaimTemplates and volumeMounts fields.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nginx
spec:
  serviceName: nginx                             # Name of the headless Service
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: container-0
          image: nginx:alpine
          resources:
            limits:
              cpu: 100m
              memory: 200Mi
            requests:
              cpu: 100m
              memory: 200Mi
          volumeMounts:                           # Storage to be mounted to the pod
          - name:  data
            mountPath:  /usr/share/nginx/html     # Mount storage to /usr/share/nginx/html.
      imagePullSecrets:
        - name: default-secret
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes:
      - ReadWriteMany
      resources:
        requests:
          storage: 1Gi
      storageClassName: csi-nas                   # Persistent StorageClass

Run the following command to create the StatefulSet:

# kubectl create -f statefulset.yaml 
statefulset.apps/nginx created

After the command is executed, check the StatefulSet and pods. The suffix of the pod names starts from 0 and increases to 2.

# kubectl get statefulset
NAME    READY   AGE
nginx   3/3     107s

# kubectl get pods
NAME      READY   STATUS    RESTARTS   AGE
nginx-0   1/1     Running   0          112s
nginx-1   1/1     Running   0          69s
nginx-2   1/1     Running   0          39s

Manually delete the nginx-1 pod and check the pods again. It is found that a pod with the same name is created. According to 5s under AGE, the nginx-1 pod is newly created.

# kubectl delete pod nginx-1
pod "nginx-1" deleted

# kubectl get pods
NAME      READY   STATUS    RESTARTS   AGE
nginx-0   1/1     Running   0          3m4s
nginx-1   1/1     Running   0          5s
nginx-2   1/1     Running   0          1m10s

Access pods and check their hostnames, which are nginx-0, nginx-1, and nginx-2.

# kubectl exec nginx-0 -- sh -c 'hostname'
nginx-0
# kubectl exec nginx-1 -- sh -c 'hostname'
nginx-1
# kubectl exec nginx-2 -- sh -c 'hostname'
nginx-2

Check the PVCs created by the StatefulSet. These PVCs are named in the format of "PVC name-StatefulSet name-No." and are in the Bound state.

# kubectl get pvc
NAME           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
data-nginx-0   Bound    pvc-f58bc1a9-6a52-4664-a587-a9a1c904ba29   1Gi        RWX            csi-nas        2m24s
data-nginx-1   Bound    pvc-066e3a3a-fd65-4e65-87cd-6c3fd0ae6485   1Gi        RWX            csi-nas        101s
data-nginx-2   Bound    pvc-a18cf1ce-708b-4e94-af83-766007250b0c   1Gi        RWX            csi-nas        71s

Network Identifier of a StatefulSet

After a StatefulSet is created, you can see that each pod has a fixed name. The headless Service provides a fixed domain name for each pod by using DNS. In this way, pods can be accessed using the domain names. Even if the IP address of a pod changes when the pod is re-built, the domain name remains unchanged.

After a headless Service is created, it allocates a domain name in the following format to each pod:

<Pod name>.<SVC name>.<Namespace>.svc.cluster.local

For example, the domain names of the preceding three pods are as follows:

• nginx-0.nginx.default.svc.cluster.local
• nginx-1.nginx.default.svc.cluster.local
• nginx-2.nginx.default.svc.cluster.local

In actual access, .<namespace>.svc.cluster.local can be omitted.

Create a pod from the tutum/dnsutils image. Then, access the container of the pod and run the nslookup command to view the domain name of the pod. The IP address of the pod can be parsed. The IP address of the DNS server is 10.247.3.10. When a CCE cluster is created, the CoreDNS add-on is installed by default to provide the DNS service. For details, see Kubernetes Networking.

$ kubectl run -i --tty --image tutum/dnsutils dnsutils --restart=Never --rm /bin/sh 
If you don't see a command prompt, try pressing enter.
/ # nslookup nginx-0.nginx
Server:         10.247.3.10
Address:        10.247.3.10#53
Name:   nginx-0.nginx.default.svc.cluster.local
Address: 172.16.0.31

/ # nslookup nginx-1.nginx
Server:         10.247.3.10
Address:        10.247.3.10#53
Name:   nginx-1.nginx.default.svc.cluster.local
Address: 172.16.0.18

/ # nslookup nginx-2.nginx
Server:         10.247.3.10
Address:        10.247.3.10#53
Name:   nginx-2.nginx.default.svc.cluster.local
Address: 172.16.0.19

Manually delete the two pods, check the IP addresses of the pods re-created by the StatefulSet, and run the nslookup command to resolve the domain names of the pods. You can still get nginx-0.nginx and nginx-1.nginx. This ensures that the network identifier of the StatefulSet remains unchanged.

StatefulSet Storage Status

As mentioned above, StatefulSets can use PVCs for persistent storage to ensure that the same persistent data can be accessed after pods are rescheduled. When pods are deleted, PVCs are not deleted.

Figure 1 Process for a StatefulSet to re-create a pod

Write data into the /usr/share/nginx/html directory of nginx-1, for example, modify the content of index.html to hello world by running the following command:

# kubectl exec nginx-1 -- sh -c 'echo hello world > /usr/share/nginx/html/index.html'

After the modification, if you access https://localhost, hello world will be returned.

# kubectl exec -it nginx-1 -- curl localhost
hello world

Manually delete the nginx-1 pod and check the pods again. It is found that a pod with the same name is created. According to 4s under AGE, the nginx-1 pod is newly created.

# kubectl delete pod nginx-1
pod "nginx-1" deleted

# kubectl get pods
NAME       READY   STATUS    RESTARTS   AGE
nginx-0    1/1     Running   0          14m
nginx-1    1/1     Running   0          4s
nginx-2    1/1     Running   0          13m

Access the index.html page of the pod again. hello world is still returned, which indicates that the same storage is accessed.

# kubectl exec -it nginx-1 -- curl localhost
hello world