Help Center/ Cloud Container Engine/ Best Practices/ Auto Scaling/ Using Knative Serving to Build a Serverless Application Platform
Updated on 2026-11-16 GMT+08:00

Using Knative Serving to Build a Serverless Application Platform

This section describes how to deploy Knative Serving v1.19 in a CCE cluster to build a serverless application platform that supports on-demand auto scaling, grayscale releases, and scaling to zero. This section covers the entire process from installation, domain name configuration, service deployment, scaling to zero verification, to grayscale release. It also provides suggestions on production environment configurations, such as KPA tuning, resource quotas, and SWR image migration.

Overview

Knative is a Kubernetes-based serverless framework that aims to provide a cross-platform serverless orchestration standard. It integrates container build (or functions), workload management, and event-driven capabilities to provide a serverless experience with declarative APIs, automatic orchestration, and traffic control on top of Kubernetes.

The following are core benefits of Knative:

  • Request-driven auto scaling (KPA): Different from HPA that is based on container metrics (CPU and memory), Knative Pod Autoscaler (KPA) dynamically adjusts the number of pods based on the number of concurrent requests or QPS. It can reduce the number of pods to zero, so that no compute resources are occupied when there is no traffic.
  • Cold starts: After the number of pods is reduced to zero, the Activator intercepts requests, triggers scale-out, and queues the requests. After the pods are ready, the Activator forwards the requests to ensure that no request is lost.
  • Version management and grayscale release: Revisions are immutable. Route uses traffic weights to implement blue-green deployment and canary release, without requiring additional toolchains.
  • Standardization: Knative is seamlessly integrated with the cloud native ecosystem based on Kubernetes CRDs.

Prerequisites

  • You have created a CCE standard or Turbo cluster v1.28 or later (v1.30 or later is recommended) with at least two nodes, and each node has at least 4 CPU cores and 8 GiB memory. A Turbo cluster (with dedicated node pools) is suitable for AI inference scenarios that require isolation.
  • You have created a load balancer for the Kourier gateway to expose services externally.
  • You have enabled SoftWare Repository for Container (SWR).
  • The nodes can access the public network, or the Knative component images have been migrated to SWR.

Constraints

  • In this example, only Knative Serving is deployed (for request-driven workload management). Knative Eventing (event-driven model) is not required. If the event-driven model (such as IoT and message queues) is required, you can add the Knative Eventing component to Knative Serving.
  • CCE does not automatically create or associate a load balancer with a Service of the LoadBalancer type. You need to manually create the load balancer and associate it with the Service using the kubernetes.io/elb.id annotation.
  • The network add-on must support NetworkPolicy. By default, CCE container networks (Cilium and Tunnel) support NetworkPolicy.
  • With scaling to zero, the time required for a cold start depends on the image pull speed and application startup duration (usually 1 to 3 seconds). For latency-sensitive workloads, set min-scale to a value to 1 or higher.
  • The services.serving.knative.dev of the Knative Service conflicts with the services of the core Service. In some CCE versions, you need to adjust the apiGroups configuration of service-webhook. For details, see Why Is the Error Message "Forbidden" Displayed When I Create a Knative Service?.

Resource Planning

  • Node planning

    Node Pool

    Specifications

    Quantity

    Purpose

    System node pool

    4C8G

    2

    Runs the Knative control plane (in the knative-serving namespace) and Kourier.

    Service node pool

    Estimated based on the peak traffic (for example, 8 CPU cores and 16 GiB of memory)

    ≥ 2

    Runs the Knative Service pods. After scaling to zero is enabled, node resources can be reused by other workloads.

    (Optional) GPU node pool

    Ascend 910 / NVIDIA GPU

    ≥ 1

    In AI inference scenarios (for example, vLLM is used to deploy large models), the required add-on must be installed.

  • Network planning

    Resource

    Plan

    Description

    ELB load balancer (public network)

    1

    The load balancer will be associated with the LoadBalancer Service of the Kourier to provide an access entry for Knative Services.

    Domain name

    1 (production environment)

    For example, a wildcard domain name *.knative.example.com is resolved to the EIP bound to the load balancer. In the test environment, sslip.io can be used for automatic resolution.

    Namespace

    knative-serving (control plane), default, or service namespace

    The control plane namespace is automatically created during installation using YAML.

  • Load balancer planning

    CCE hws-cloudprovider does not automatically create or associate a load balancer for a Service of the LoadBalancer type. You must manually create a load balancer and explicitly associate it with the Service using the Service annotation.

    • Load balancer type selection

      Type

      elb.class

      Application Scenario

      Quota

      Shared load balancer

      union

      Test environments and low-traffic scenarios

      The listener and backend server group quotas are small. When multiple Services share the quotas, the error message "Quota exceeded for resources: pool" may be displayed.

      Dedicated load balancer

      performance

      Production environments and high-traffic scenarios

      Independent resource quota and performance isolation. This type of load balancer is suitable for Knative production deployment.

    • Create a load balancer.
      • Type: Select the shared load balancer for the test environment and dedicated load balancer for the production environment.
      • Network configuration: Select the VPC that is the same as that of the current CCE cluster.
      • Record the load balancer ID.

        You do not need to manually configure the listener and backend server group. hws-cloudprovider automatically creates them based on spec.ports of the Service. If the listener and backend server group is left over due to a previous failed apply operation, you need to manually delete the listener or pool on the ELB console and then perform the apply operation again.

    • Bind the load balancer to the Kourier Service.
      • Add the following annotations to the Kourier Service (the Kourier Service in the kourier-system namespace):
        apiVersion: v1
        kind: Service
        metadata:
          name: kourier
          namespace: kourier-system
          annotations:
            kubernetes.io/elb.id: <your-elb-id>        # Replace it with the actual load balancer ID.
            kubernetes.io/elb.class: union              # Enter union for the shared load balancer and performance for the dedicated load balancer.
        spec:
          type: LoadBalancer
          #... Keep other fields unchanged.
      • Run the following kubectl command:
        kubectl annotate svc kourier -n kourier-system \
          kubernetes.io/elb.id=<your-elb-id> \
          kubernetes.io/elb.class=union                 # Change the value to performance is a dedicated load balancer is used.

        Note: If the association fails, check the following common errors:

        • "listener is empty" or "elb.id is not defined, skip": The Service lacks the elb.id annotation, and hws-cloudprovider cannot create a listener.
        • "Quota exceeded for resources: pool (ELB.8905)": The backend server group quota of the shared load balancer has been used up. You need to delete unnecessary backend server groups or replace the shared load balancer with a dedicated load balancer.
  • Image repository planning

    By default, Knative component images are obtained from gcr.io/knative-releases and ghcr.io/knative. If CCE nodes cannot directly pull the images over the public network, you need to migrate the images to SWR. For details about how to migrate images, see Pushing an Image.

    Component

    Default Image

    SWR Target

    Activator/Autoscaler/Controller/Webhook

    gcr.io/knative-releases/knative.dev/serving/cmd/*
    swr.<region>.myhuaweicloud.com/<project>/knative-*:v1.19.6

    net-kourier-controller/kourier-gateway

    gcr.io/knative-releases/knative.dev/net-kourier/cmd/*
    swr.<region>.myhuaweicloud.com/<project>/kourier-*:v1.19.0

Procedure

Step 1: Install kn.

kn is the command line client of Knative. It is used to create, update, and delete Knative Services and manage revisions.

  1. Use kubectl to connect to the cluster. For details, see Using kubectl to Access a Cluster.
  2. Run the command below to install kn. Knative v1.19.0 is used as an example.
    # Linux amd64
    curl -Lo ./kn https://github.com/knative/client/releases/download/knative-v1.19.0/kn-linux-amd64
    chmod +x ./kn
    sudo mv ./kn /usr/local/bin/
  3. Verify the installation.
    kn version

    Expected output:

    Version:      v1.19.0
    Build Date:   2026-xx-xx
    Git Revision: xxxxxxx
    Supported APIs:
    * Serving v1 (knative-serving v1.19.x)

Step 2: Install the CRDs and core components of Knative Serving.

If nodes cannot pull images over the public network, migrate the images to SWR and replace image path in the following YAML file with the SWR address:

  1. Install the CRDs.
    kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.19.6/serving-crds.yaml
    kubectl wait --for=condition=Established --all crd --timeout=120s

    The expected output contains the following information:

    customresourcedefinition.apiextensions.k8s.io/services.serving.knative.dev created
    customresourcedefinition.apiextensions.k8s.io/configurations.serving.knative.dev created
    customresourcedefinition.apiextensions.k8s.io/revisions.serving.knative.dev created
    customresourcedefinition.apiextensions.k8s.io/routes.serving.knative.dev created
    ...
  2. Install the core components.
    kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.19.6/serving-core.yaml

    The components are deployed in the knative-serving namespace using the YAML file.

    Component

    Description

    Controller

    Listens to changes of Service/Configuration CRDs and coordinates the generation of revisions and Deployments.

    Webhook

    Verifies the CRD validity and injects default values.

    Autoscaler

    Collects pod concurrency metrics, adjusts the number of replicas in the Deployments, and supports scaling down to 0.

    Activator

    Intercepts requests when the number of pods is scaled down to 0, triggers scale-out, and forwards the requests in a queue.

  3. Run the following command to check whether all pods in the knative-serving namespace are in the Running state:
    kubectl get pods -n knative-serving

    Expected output:

    NAME                                READY   STATUS    RESTARTS   AGE
    activator-xxxxx-xxxxx               1/1     Running   0          60s
    autoscaler-xxxxx-xxxxx              1/1     Running   0          60s
    controller-xxxxx-xxxxx              1/1     Running   0          60s
    webhook-xxxxx-xxxxx                 1/1     Running   0          60s

    If the Webhook is in the CrashLoopBackOff state, it is mostly because the TLS certificate is not ready. Wait for 1 to 2 minutes until the Webhook automatically recovers. Knative v1.19 has built-in certificates. You do not need to deploy cert-manager.

Step 3: Deploy the Kourier and Configure an Ingress

Knative supports multiple types of ingresses (Kourier, Istio, and Contour). Kourier is a lightweight ingress based on Envoy. It has few dependencies and can be started quickly. It is recommended as the default gateway for beginners and production.

  1. Deploy Kourier.
    kubectl apply -f https://github.com/knative/net-kourier/releases/download/knative-v1.19.0/kourier.yaml

    Kourier is deployed in the kourier-system namespace and contains 3scale-kourier-gateway (Envoy on the data plane). The ingress is exposed through the LoadBalancer Service. CCE does not automatically associate the load balancer with the Kourier Service and allocate an external IP address. You need to manually associate the created load balancer with the Kourier Service using an annotation. If the ELB annotation is not configured, hws-cloudprovider reports the error message "listener is empty" or "elb.id is not defined, skip".

  2. Configure Kourier as the default ingress.
    kubectl patch configmap/config-network \
      --namespace knative-serving \
      --type merge \
      --patch '{"data":{"ingress-class":"kourier.ingress.networking.knative.dev"}}'
  3. Verify that the ingress is ready and obtain the access IP address.
    kubectl get pods -n kourier-system
    kubectl get svc kourier -n kourier-system

    Expected output:

    NAME                             READY   STATUS    RESTARTS   AGE
    3scale-kourier-control-xxxxx    1/1     Running   0          60s
    3scale-kourier-gateway-xxxxx     1/1     Running   0          60s
    
    NAME      TYPE           CLUSTER-IP    EXTERNAL-IP      PORT(S)                      AGE
    kourier   LoadBalancer   10.x.x.x      121.xxx.xxx.xxx   80:3xx/TCP,443:3xx/TCP        90s

    EXTERNAL-IP is the public or private IP address bound to the load balancer. You need to associate the load balancer in advance using the kubernetes.io/elb.id annotation. Record this IP address, which will be used for domain name resolution.

Step 4: Configure the domain name and DNS.

The default domain name suffix of Knative is determined by the config-domain ConfigMap. The access address of all Knative Services is <service-name>.<namespace>.<domain>.

  • Test environment: Use sslip.io for automatic resolution.

    sslip.io resolves {ip}.sslip.io to {ip}. You do not need to set up DNS. Use the load balancer's IP address obtained in the previous step to combine a domain name.

    INGRESS_IP=$(kubectl get svc kourier -n kourier-system -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
    KNATIVE_DOMAIN="${INGRESS_IP}.sslip.io"
    
    kubectl patch configmap/config-domain \
      --namespace knative-serving \
      --type merge \
      --patch "{\"data\":{\"${KNATIVE_DOMAIN}\":\"\"}}"
    
    echo "Knative domain: ${KNATIVE_DOMAIN}"

    The expected output is as follows:

    Knative domain: 121.xxx.xxx.xxx.sslip.io
  • Production environment: Use a custom domain name.
    1. Resolve the wildcard domain name *.knative.<your-domain> to the EIP bound to the load balancer.
    2. Configure config-domain.
      kubectl patch configmap/config-domain \
        --namespace knative-serving \
        --type merge \
        --patch '{"data":{"knative.example.com":""}}'
    3. (Optional) Configure a TLS certificate for port 443 of the Kourier. You are advised to use CCE certificate management or secret injection for HTTPS access.

Step 5: Deploy the first Knative Service.

  1. Use Knative to deploy helloworld.
    kn service create helloworld \
      --image ghcr.io/knative/helloworld-go:latest \
      --port 8080 \
      --env TARGET=World

    If the nodes cannot pull images from ghcr.io, migrate the images to SWR and replace them with swr.<region>.myhuaweicloud.com/<project>/helloworld-go:latest.

    Expected output:

    Creating service 'helloworld' in namespace 'default':
      0.030s The Route is still working to reflect the latest desired specification.
      0.062s ...
      0.080s Configuration "helloworld" is waiting for a Revision to become ready.
      8.156s ...
      8.178s Ingress has not yet been reconciled.
      8.298s Ready to serve.
    Service 'helloworld' created with latest revision 'helloworld-00001' and URL:
    http://helloworld.default.121.xxx.xxx.xxx.sslip.io
  2. View the Service and underlying resources.
    kn service list
    kubectl get configuration,revision,route,deployment -l serving.knative.dev/service=helloworld

    Expected output:

    NAME            URL                                                LATEST CREATED   LATEST READY   READY   REASON
    helloworld      http://helloworld.default.121.xxx.xxx.xxx.sslip.io  helloworld-00001 helloworld-00001 True
    
    NAME                                                  READY   STATUS
    configuration.serving.knative.dev/helloworld          True
    revision.serving.knative.dev/helloworld-00001         True
    route.serving.knative.dev/helloworld                  True
    deployment.apps/helloworld-00001-deployment           1/1

    According to the auto scaling mechanism of Knative Serving, when there is no incoming traffic, the number of Deployment replicas corresponding to the revision will be automatically reduced to 0 (that is, 0/0).

    Knative Service automatically creates four levels of resources: Configuration, Revision, Route, and Deployment. The CRD hierarchy is as follows: Service → Configuration → Revision → Deployment.

  3. Access the Service. Test the helloworld Service on any node or machine that can access the public network.
    curl http://helloworld.default.${KNATIVE_DOMAIN}

    If "Hello World!" is returned, the Service is successfully deployed.

Step 6: Verify scaling to zero.

Scale to zero is a core feature of Knative. When there is no traffic, the Autoscaler reduces the number of Deployment replicas to 0. When a request arrives, the Activator intercepts the request and triggers scale-out.

  1. Observe the scale-down to zero.

    After the Service has no requests for a period of time (the default idle window is 30 seconds), the pod is automatically terminated.

    # Continuously observe the pod status.
    kubectl get pod -l serving.knative.dev/service=helloworld -w

    Expected result: After about 30 to 60 seconds, the number of pods changes to 0.

    NAME                                         READY   STATUS    RESTARTS   AGE
    helloworld-00001-deployment-xxxxx-xxxxx      2/2     Running   0          60s
    helloworld-00001-deployment-xxxxx-xxxxx      2/1     Terminating   0      90s
    helloworld-00001-deployment-xxxxx-xxxxx      0/1     Terminating   0      95s
  2. Trigger a cold start.

    When the number of pods is 0 and a request is initiated, the Activator intercepts the request, triggers scale-out, and forwards the request.

    curl -w "\nTotal time: %{time_total}s\n" http://helloworld.default.${KNATIVE_DOMAIN}

    Expected output:

    Hello World!
    Total time: 1.2s

    The duration of the first cold start depends on the image pull and application startup duration (usually 1 to 3 seconds). Subsequent requests will directly hit the running pod, reducing the time to milliseconds.

  3. (Optional) Adjust the scale-down window.

    By default, Knative reduces the number of pods to 0 after the pods are idle for 30 seconds. You can adjust the window by using the revision annotation, for example, extending the window to 1 minute.

    kn service update helloworld \
      --annotation autoscaling.knative.dev/scale-to-zero-pod-retention-period=1m

    Expected output:

    Updating service 'helloworld' in namespace 'default':
      0.030s The Route is still working to reflect the latest desired specification.
      0.062s ...
      0.080s Configuration "helloworld" is waiting for a Revision to become ready.
      1.156s ...
      1.178s Ingress has not yet been reconciled.
      1.298s Ready to serve.
    Service 'helloworld' updated to latest revision 'helloworld-00003' and URL:
    http://helloworld.default.121.xxx.xxx.xxx.sslip.io

    Modifying the annotation will trigger Knative to create a new revision (for example, helloworld-00003 in the example). The old revision will gradually stop receiving traffic, and the new revision will handle subsequent requests.

Step 7: Verify the grayscale release (blue-green deployment or canary release).

Knative implements grayscale releases using immutable revisions and traffic weights, without the need to introduce additional toolchains.

  1. Update the image. (A new revision is automatically generated.)
    kn service update helloworld \
      --image ghcr.io/knative/helloworld-go:latest \
      --env TARGET=Knative

    This operation will automatically create a new revision (helloworld-00002), and the default route will switch 100% traffic to the new version. The old revision (helloworld-00001) is retained but does not receive traffic.

  2. Canary release: Traffic is distributed based on weights.

    The traffic is split: 90% to the old version and 10% to the new version. After the observation, the traffic is gradually switched.

    kn service update helloworld \
      --traffic helloworld-00001=90 \
      --traffic helloworld-00002=10

    The --traffic parameter is in the format of Revision name=Percentage. The total weight of multiple revisions must be 100.

  3. Verify the traffic distribution. Request the Service multiple times and observe the distribution of returned content.
    for i in $(seq 1 10); do
      curl -s http://helloworld.default.${KNATIVE_DOMAIN}
    done | sort | uniq -c

    Expected output (about 9:1):

    9 Hello World!
    1 Hello Knative!
  4. View route details.
    kn route describe helloworld

    Expected output:

    Name:       helloworld
    Namespace:  default
    Age:        10m
    URL:        http://helloworld.default.121.xxx.xxx.xxx.sslip.io
    
    Traffic Targets:
      90%  @latest  (helloworld-00001) # still old
            URL:  http://helloworld.default.../helloworld-00001
      10%  helloworld-00002
            URL:  http://helloworld.default.../helloworld-00002
  5. Roll back the version.

    A revision is an immutable snapshot. The rollback operation redirects the traffic to the old revision based on the weights.

    kn service update helloworld --traffic helloworld-00001=100

    After the command is executed, the traffic is immediately switched back to helloworld-00001, and no redeployment is required.

Configuration Optimization

KPA Auto Scaling Optimization

By default, KPA performs auto scaling based on the number of concurrent requests. In the production environment, you need to adjust key parameters based on service characteristics.

Key parameters are configured using template.metadata.annotations of the revision.

Annotation

Default Value

Description

Suggestions

autoscaling.knative.dev/metric

concurrency

Scaling metric

Set the value to rps for short tasks with high QPS and to concurrency for persistent connection or recalculation tasks.

autoscaling.knative.dev/target

100 (concurrency)/200 (rps)

Target concurrency/Rate per pod

Set this parameter based on the single-pod bearing capability. If the value is too low, excessive scale-out will be performed. If the value is too high, traffic limiting will be triggered.

autoscaling.knative.dev/max-scale

0 (no upper limit)

Maximum number of pods

This parameter is mandatory in the production environment to prevent the node pool resources from being used up due to sudden traffic increase.

autoscaling.knative.dev/min-scale

0

Minimum number of pods

Set this parameter to 1 or 2 for latency-sensitive workloads to reserve pods to be kept warm.

autoscaling.knative.dev/scale-to-zero-pod-retention-period

0s

Retention duration before scaling down to 0

Set this parameter to 30s to 1 minute to smoothly handle intermittent traffic.

autoscaling.knative.dev/window

60s

Scaling calculation window

In burst traffic scenarios, the value is shortened to 30s. In stable traffic scenarios, the value is prolonged to 120s to reduce jitter.

The annotations in this table apply to a single revision. To set global default values for all Services, edit config-autoscaler ConfigMap: kubectl edit configmap/config-autoscaler -n knative-serving. The annotation priority is higher than the global default values.

Example: Configuring KPA for a high-QPS web service

The following uses the helloworld Service as an example to describe how to configure KPA parameters for high-QPS scenarios:

kn service create web-api \
  --image swr.cn-north-7.myhuaweicloud.com/<project>/web-api:latest \
  --port 8080 \
  --annotation autoscaling.knative.dev/metric=rps \
  --annotation autoscaling.knative.dev/target=200 \
  --annotation autoscaling.knative.dev/max-scale=20 \
  --annotation autoscaling.knative.dev/min-scale=2 \
  --annotation autoscaling.knative.dev/window=30s

Resource Quotas

To prevent the control plane from being overloaded, pod scheduling failures, or cluster resources from being exhausted, you are advised to configure resource quotas based on the policies below.

Configuration Item

Recommended Value

Description

Control plane pod resource requests

Controller/Autoscaler: 500m/512Mi; Webhook 100m/50Mi

The default values are low. You are advised to increase the values for production workloads.

Service pod requests

Set the values based on the bearer capability of a single pod.

If the values are too low, node resources may be overcommitted. If the values are too high, scheduling may fail. By default, Knative calculates the number of concurrent requests based on this configuration item.

Service pod limits

2 to 4 times of service pod requests

Reserve resources for traffic bursts to prevent OOM caused by traffic bursts. In the GPU scenario, the limits must be equal to requests.

namespace ResourceQuota

Set the values based on the requirements of the service namespace.

Limit the resource quotas of a single namespace to prevent cluster resources from being exhausted or services from being unavailable due to unlimited scale-out of Knative Services.

Adjust the default resource policy globally through the config-resource ConfigMap:

kubectl patch configmap/config-resource \
  --namespace knative-serving \
  --type merge \
  --patch '{"data":{"pod-container-default-requests":"{\"cpu\":\"500m\",\"memory\":\"512Mi\"}"}}'

Common Issues