StatefulSet

StatefulSet

All pods under a Deployment have the same characteristics except for the name and IP address. If required, a Deployment can use the pod template to create a new pod. If not required, the Deployment can delete any one of the pods.

However, Deployments cannot meet the requirements in some distributed scenarios when each pod requires its own status or in a distributed database where each pod requires independent storage.

With detailed analysis, it is found that each part of distributed stateful applications plays a different role. For example, the database nodes are deployed in active/standby mode, and pods are dependent on each other. In this case, you need to meet the following requirements for the pods:

• A pod can be recognized by other pods. Therefore, a pod must have a fixed identifier.
• Each pod has an independent storage device. After a pod is deleted and then restored, the data read from the pod must be the same as the previous one. Otherwise, the pod status is inconsistent.

To address the preceding requirements, Kubernetes provides StatefulSets.

1. A StatefulSet provides a fixed name for each pod following a fixed number ranging from 0 to N. After a pod is rescheduled, the pod name and the host name remain unchanged.
2. A StatefulSet provides a fixed access domain name for each pod through the headless Service (described in following sections).
3. The StatefulSet creates PersistentVolumeClaims (PVCs) with fixed identifiers to ensure that pods can access the same persistent data after being rescheduled.

   

Creating a Headless Service

As described above, a headless Service is required for pod access when a StatefulSet is created. For details about the Service, see Service. The following describes how to create a headless Service.

Use the following file to describe the headless Service:

• spec.clusterIP: Set it to None, which indicates a headless Service is to be created.
• spec.ports.port: indicates the number of the port used for communication between pods.
• spec.ports.name: indicates the name of the port used for communication between pods.

apiVersion: v1
kind: Service       # The object type is Service.
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
    - name: nginx     # Name of the port for communication between pods
      port: 80        # Number of the port for communication between pods
  selector:
    app: nginx        # Select the pod whose label is app:nginx. 
  clusterIP: None     # Set this parameter to None, indicating that a headless Service is to be created.

Run the following command to create a headless Service:

# kubectl create -f headless.yaml 
service/nginx created

After the Service is created, you can query the Service information.

# kubectl get svc
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
nginx        ClusterIP   None         <none>        80/TCP    5s

Creating a StatefulSet

The YAML definition of StatefulSets is basically the same as that of other objects. The differences are as follows:

• serviceName specifies the headless Service used by the StatefulSet. You need to specify the name of the headless Service.
• volumeClaimTemplates is used to apply for a PVC. A template named data is defined, which will create a PVC for each pod. storageClassName specifies the persistent storage class. For details, see PersistentVolume, PersistentVolumeClaim, and StorageClass. volumeMounts is used to mount storage to pods. If no storage is required, you can delete the volumeClaimTemplates and volumeMounts fields.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nginx
spec:
  serviceName: nginx                             # Name of the headless Service
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: container-0
          image: nginx:alpine
          resources:
            limits:
              cpu: 100m
              memory: 200Mi
            requests:
              cpu: 100m
              memory: 200Mi
          volumeMounts:                           # Storage mounted to the pod
          - name:  data
            mountPath: /usr/share/nginx/html      # Mount the storage to /usr/share/nginx/html.
      imagePullSecrets:
        - name: default-secret
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes:
      - ReadWriteMany
      resources:
        requests:
          storage: 1Gi
      storageClassName: csi-nas                   # Persistent storage class

Run the following command to create a StatefulSet:

# kubectl create -f statefulset.yaml 
statefulset.apps/nginx created

After the command is executed, query the StatefulSet and pods. The suffix of the pod names starts from 0 and increases to 2.

# kubectl get statefulset
NAME    READY   AGE
nginx   3/3     107s

# kubectl get pods
NAME      READY   STATUS    RESTARTS   AGE
nginx-0   1/1     Running   0          112s
nginx-1   1/1     Running   0          69s
nginx-2   1/1     Running   0          39s

In this case, if you manually delete the nginx-1 pod and query the pods again, you can see that a pod with the same name is created. According to 5s under AGE, it is found that the nginx-1 pod is newly created.

# kubectl delete pod nginx-1
pod "nginx-1" deleted

# kubectl get pods
NAME      READY   STATUS    RESTARTS   AGE
nginx-0   1/1     Running   0          3m4s
nginx-1   1/1     Running   0          5s
nginx-2   1/1     Running   0          1m10s

Access the container and check its host names. The host names are nginx-0, nginx-1, and nginx-2.

# kubectl exec nginx-0 -- sh -c 'hostname'
nginx-0
# kubectl exec nginx-1 -- sh -c 'hostname'
nginx-1
# kubectl exec nginx-2 -- sh -c 'hostname'
nginx-2

In addition, you can view the PVCs created by the StatefulSet. These PVCs are named in the format of PVC name-StatefulSet name-No. and are in the Bound state.

# kubectl get pvc
NAME           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
data-nginx-0   Bound    pvc-f58bc1a9-6a52-4664-a587-a9a1c904ba29   1Gi        RWX            csi-nas        2m24s
data-nginx-1   Bound    pvc-066e3a3a-fd65-4e65-87cd-6c3fd0ae6485   1Gi        RWX            csi-nas        101s
data-nginx-2   Bound    pvc-a18cf1ce-708b-4e94-af83-766007250b0c   1Gi        RWX            csi-nas        71s

Network Identifier of a StatefulSet

After a StatefulSet is created, you can see that each pod has a fixed name. The headless Service provides a fixed domain name for pods by using DNS. In this way, pods can be accessed using the domain name. Even if the IP address of the pod changes when the pod is re-created, the domain name remains unchanged.

After a headless Service is created, the IP address of each pod corresponds to a domain name in the following format:

<pod-name>.<svc-name>.<namespace>.svc.cluster.local

For example, the domain names of the three pods are as follows:

• nginx-0.nginx.default.svc.cluster.local
• nginx-1.nginx.default.svc.cluster.local
• nginx-2.nginx.default.svc.cluster.local

In actual access, .<namespace>.svc.cluster.local can be omitted.

Create a pod from the tutum/dnsutils image. Then, access the container of the pod and run the nslookup command to view the domain name of the pod. The IP address of the pod can be parsed. The IP address of the DNS server is 10.247.3.10. When a CCE cluster is created, the coredns add-on is installed by default to provide the DNS service. The functions of coredns will be described in Kubernetes Networking.

$ kubectl run -i --tty --image tutum/dnsutils dnsutils --restart=Never --rm /bin/sh 
If you don't see a command prompt, try pressing enter.
/ # nslookup nginx-0.nginx
Server:         10.247.3.10
Address:        10.247.3.10#53
Name:   nginx-0.nginx.default.svc.cluster.local
Address: 172.16.0.31

/ # nslookup nginx-1.nginx
Server:         10.247.3.10
Address:        10.247.3.10#53
Name:   nginx-1.nginx.default.svc.cluster.local
Address: 172.16.0.18

/ # nslookup nginx-2.nginx
Server:         10.247.3.10
Address:        10.247.3.10#53
Name:   nginx-2.nginx.default.svc.cluster.local
Address: 172.16.0.19

In this case, if you manually delete the two pods, query the IP addresses of the pods re-created by the StatefulSet, and run the nslookup command to resolve the domain names of the pods, you can still get nginx-0.nginx and nginx-1.nginx. This ensures that the network identifier of the StatefulSet remains unchanged.

StatefulSet Storage Status

As mentioned above, StatefulSets can use PVCs for persistent storage to ensure that the same persistent data can be accessed after pods are rescheduled. When pods are deleted, PVCs are not deleted.

Figure 1 Process for a StatefulSet to re-create a pod

Run the following command to write some data into the /usr/share/nginx/html directory of nginx-1. For example, change the content of index.html to hello world.

# kubectl exec nginx-1 -- sh -c 'echo hello world > /usr/share/nginx/html/index.html'

After the modification, if you access https://localhost, hello world is returned.

# kubectl exec -it nginx-1 -- curl localhost
hello world

In this case, if you manually delete the nginx-1 pod and query the pods again, you can see that a pod with the same name is created. According to 4s under AGE, it is found that the nginx-1 pod is newly created.

# kubectl delete pod nginx-1
pod "nginx-1" deleted

# kubectl get pods
NAME       READY   STATUS    RESTARTS   AGE
nginx-0    1/1     Running   0          14m
nginx-1    1/1     Running   0          4s
nginx-2    1/1     Running   0          13m

Access the index.html page of the pod again. hello world is still returned, which indicates that the same storage medium is accessed.

# kubectl exec -it nginx-1 -- curl localhost
hello world