Configuring Grayscale Release for a LoadBalancer Ingress

Notes and Constraints

• This feature takes effect only in the following versions:
  • v1.23: v1.23.14-r0 or later
  • v1.25: v1.25.9-r0 or later
  • v1.27: v1.27.6-r0 or later
  • v1.28: v1.28.4-r0 or later
  • Other clusters of later versions
• After a grayscale ingress is created, do not delete the original ingress.
• If the listeners of a load balancer associate with multiple ingresses configured with multiple grayscale release policies, the header-based policy has the highest priority, cookie has the second highest priority, and weight has the lowest priority.

Parameters

{"persistence_timeout": "1440"}

'{"values":["a","b"]}'

'{"values":["a","b"]}'

Deploying Grayscale Release

1. Deploy the original service.
   1. Deploy a workload named origin-server.
   2. Deploy a Service named origin-service and associate it with the created origin-server workload. (Create a NodePort Service in a CCE standard cluster or a ClusterIP Service in a CCE Turbo cluster.)
   3. Deploy an ingress named origin-ingress and associate it with the created origin-service Service.
      Configure the ingress as follows:

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: origin-ingress
        namespace: default
        annotations:
          kubernetes.io/elb.port: '81'
          kubernetes.io/elb.id: e491f4e7-2351-4984-ad0a-8569e5e964a3
          kubernetes.io/elb.class: performance
      spec:
        rules:
          - host: nginx1.com
            http:
              paths:
                - path: /
                  backend:
                    service:
                      name: origin-service
                      port:
                        number: 81
                  property:
                    ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH
                  pathType: ImplementationSpecific
        ingressClassName: cce

2. Grayscale release a new version for the Service.

   Deploy the workload, Service, and ingress of a new version so that traffic can be routed to the Service of the new version by weight or header.

   1. Deploy a workload named canary-server.
   2. Deploy a Service named canary-service and associate it with the created canary-server workload. (Create a NodePort Service in a CCE standard cluster or a ClusterIP Service in a CCE Turbo cluster.)
   3. Deploy an ingress named canary-weight-ingress and associate it with the created canary-service Service for grayscale release by weight.
      Configure the ingress as follows:

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: canary-weight-ingress
        namespace: default
        annotations:
          kubernetes.io/elb.canary: 'true'             // Set this parameter to true, indicating that canary annotation is enabled.
          kubernetes.io/elb.canary-weight: '40'        // Configure a weight, indicating that 40% of the request traffic will be routed to the Service of the new version.
          kubernetes.io/elb.class: performance         // Only dedicated load balancers support grayscale release.
          kubernetes.io/elb.id: e491f4e7-2351-4984-ad0a-8569e5e964a3
          kubernetes.io/elb.port: '81'
      spec:
        ingressClassName: cce
        rules:
          - host: nginx1.com
            http:
              paths:
                - path: /
                  pathType: ImplementationSpecific
                  backend:
                    service:
                      name: canary-service
                      port:
                        number: 81
                  property:
                    ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH

   4. Deploy an ingress named canary-header-ingress and associate it with the created canary-service Service for grayscale release by header.
      Configure the ingress as follows:

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: canary-header-ingress
        namespace: default
        annotations:
          kubernetes.io/elb.canary: 'true'
          kubernetes.io/elb.canary-by-header: 'location'                           // Set the header key to location.
          kubernetes.io/elb.canary-by-header-value: '{"values":["hz","sz","sh"]}'  // Set the header values to hz, sz, and sh.
          kubernetes.io/elb.class: performance
          kubernetes.io/elb.id: e491f4e7-2351-4984-ad0a-8569e5e964a3
          kubernetes.io/elb.port: '81'
      spec:
        ingressClassName: cce
        rules:
          - host: nginx1.com
            http:
              paths:
                - path: /
                  pathType: ImplementationSpecific
                  backend:
                    service:
                      name: canary-service
                      port:
                        number: 80
                  property:
                    ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH

   5. Check whether the new version is grayscale released.
      1. Check the creation of the ingress.

         $ kubectl get ingress
         NAME                    CLASS   HOSTS        ADDRESS        PORTS   AGE
         canary-header-ingress   cce     nginx1.com   88.88.88.165   80      46s
         canary-weight-ingress   cce     nginx1.com   88.88.88.165   80      117s
         origin-ingress          cce     nginx1.com   88.88.88.165   80      2m25s

      2. Check ELB rules.

         

      3. Check the result of a service request.
         1. Use the location: hz request header to access the Service.

            $ curl -H "Host: nginx1.com" -H "location: hz" http://88.88.88.165:81/

         2. Expected result:

            $ new

            Attempt the access for multiple times. All the returned results are new.

         3. Access the Service without using a request header.

            $ curl -H "Host: nginx1.com" http://88.88.88.165:81/

         4. Expected result: 40% for new, and 60% for old

3. Terminate the old-version Service.

   After the Service of the new version runs stably, terminate the Service of the old version. During the Service termination, change the original Service to the new-version Service so that all traffic will be routed to the new-version Service. Then, delete the grayscale released ingress.

   1. Change the original Service to the new-version Service.

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: origin-ingress
        namespace: default
        annotations:
          kubernetes.io/elb.port: '81'
          kubernetes.io/elb.id: e491f4e7-2351-4984-ad0a-8569e5e964a3
          kubernetes.io/elb.class: performance
      spec:
        rules:
          - host: nginx1.com
            http:
              paths:
                - path: /
                  backend:
                    service:
                      name: canary-service
                      port:
                        number: 81
                  property:
                    ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH
                  pathType: ImplementationSpecific
        ingressClassName: cce

   2. Check whether the old-version Service has been terminated.
      1. Initiate a request with a header and a request without a header to access the Service.

         $ curl -H "Host: nginx1.com" -H "location: hz" http://88.88.88.165:81/
         $ curl -H "Host: nginx1.com" http://88.88.88.165:81/

      2. Expected result: All results are returned from the new-version Service.
   3. Delete the grayscale released ingress.

      $ kubectl delete ingress canary-weight-ingress canary-header-ingress