Using Knative Serving to Build a Serverless Application Platform
This section describes how to deploy Knative Serving v1.19 in a CCE cluster to build a serverless application platform that supports on-demand auto scaling, grayscale releases, and scaling to zero. This section covers the entire process from installation, domain name configuration, service deployment, scaling to zero verification, to grayscale release. It also provides suggestions on production environment configurations, such as KPA tuning, resource quotas, and SWR image migration.
Overview
Knative is a Kubernetes-based serverless framework that aims to provide a cross-platform serverless orchestration standard. It integrates container build (or functions), workload management, and event-driven capabilities to provide a serverless experience with declarative APIs, automatic orchestration, and traffic control on top of Kubernetes.
The following are core benefits of Knative:
- Request-driven auto scaling (KPA): Different from HPA that is based on container metrics (CPU and memory), Knative Pod Autoscaler (KPA) dynamically adjusts the number of pods based on the number of concurrent requests or QPS. It can reduce the number of pods to zero, so that no compute resources are occupied when there is no traffic.
- Cold starts: After the number of pods is reduced to zero, the Activator intercepts requests, triggers scale-out, and queues the requests. After the pods are ready, the Activator forwards the requests to ensure that no request is lost.
- Version management and grayscale release: Revisions are immutable. Route uses traffic weights to implement blue-green deployment and canary release, without requiring additional toolchains.
- Standardization: Knative is seamlessly integrated with the cloud native ecosystem based on Kubernetes CRDs.
Prerequisites
- You have created a CCE standard or Turbo cluster v1.28 or later (v1.30 or later is recommended) with at least two nodes, and each node has at least 4 CPU cores and 8 GiB memory. A Turbo cluster (with dedicated node pools) is suitable for AI inference scenarios that require isolation.
- You have created a load balancer for the Kourier gateway to expose services externally.
- You have enabled SoftWare Repository for Container (SWR).
- The nodes can access the public network, or the Knative component images have been migrated to SWR.
Constraints
- In this example, only Knative Serving is deployed (for request-driven workload management). Knative Eventing (event-driven model) is not required. If the event-driven model (such as IoT and message queues) is required, you can add the Knative Eventing component to Knative Serving.
- CCE does not automatically create or associate a load balancer with a Service of the LoadBalancer type. You need to manually create the load balancer and associate it with the Service using the kubernetes.io/elb.id annotation.
- The network add-on must support NetworkPolicy. By default, CCE container networks (Cilium and Tunnel) support NetworkPolicy.
- With scaling to zero, the time required for a cold start depends on the image pull speed and application startup duration (usually 1 to 3 seconds). For latency-sensitive workloads, set min-scale to a value to 1 or higher.
- The services.serving.knative.dev of the Knative Service conflicts with the services of the core Service. In some CCE versions, you need to adjust the apiGroups configuration of service-webhook. For details, see Why Is the Error Message "Forbidden" Displayed When I Create a Knative Service?.
Resource Planning
- Node planning
Node Pool
Specifications
Quantity
Purpose
System node pool
4C8G
2
Runs the Knative control plane (in the knative-serving namespace) and Kourier.
Service node pool
Estimated based on the peak traffic (for example, 8 CPU cores and 16 GiB of memory)
≥ 2
Runs the Knative Service pods. After scaling to zero is enabled, node resources can be reused by other workloads.
(Optional) GPU node pool
Ascend 910 / NVIDIA GPU
≥ 1
In AI inference scenarios (for example, vLLM is used to deploy large models), the required add-on must be installed.
- Network planning
Resource
Plan
Description
ELB load balancer (public network)
1
The load balancer will be associated with the LoadBalancer Service of the Kourier to provide an access entry for Knative Services.
Domain name
1 (production environment)
For example, a wildcard domain name *.knative.example.com is resolved to the EIP bound to the load balancer. In the test environment, sslip.io can be used for automatic resolution.
Namespace
knative-serving (control plane), default, or service namespace
The control plane namespace is automatically created during installation using YAML.
- Load balancer planning
CCE hws-cloudprovider does not automatically create or associate a load balancer for a Service of the LoadBalancer type. You must manually create a load balancer and explicitly associate it with the Service using the Service annotation.
- Load balancer type selection
Type
elb.class
Application Scenario
Quota
Shared load balancer
union
Test environments and low-traffic scenarios
The listener and backend server group quotas are small. When multiple Services share the quotas, the error message "Quota exceeded for resources: pool" may be displayed.
Dedicated load balancer
performance
Production environments and high-traffic scenarios
Independent resource quota and performance isolation. This type of load balancer is suitable for Knative production deployment.
- Create a load balancer.
- Type: Select the shared load balancer for the test environment and dedicated load balancer for the production environment.
- Network configuration: Select the VPC that is the same as that of the current CCE cluster.
- Record the load balancer ID.
You do not need to manually configure the listener and backend server group. hws-cloudprovider automatically creates them based on spec.ports of the Service. If the listener and backend server group is left over due to a previous failed apply operation, you need to manually delete the listener or pool on the ELB console and then perform the apply operation again.
- Bind the load balancer to the Kourier Service.
- Add the following annotations to the Kourier Service (the Kourier Service in the kourier-system namespace):
apiVersion: v1 kind: Service metadata: name: kourier namespace: kourier-system annotations: kubernetes.io/elb.id: <your-elb-id> # Replace it with the actual load balancer ID. kubernetes.io/elb.class: union # Enter union for the shared load balancer and performance for the dedicated load balancer. spec: type: LoadBalancer #... Keep other fields unchanged. - Run the following kubectl command:
kubectl annotate svc kourier -n kourier-system \ kubernetes.io/elb.id=<your-elb-id> \ kubernetes.io/elb.class=union # Change the value to performance is a dedicated load balancer is used.
Note: If the association fails, check the following common errors:
- "listener is empty" or "elb.id is not defined, skip": The Service lacks the elb.id annotation, and hws-cloudprovider cannot create a listener.
- "Quota exceeded for resources: pool (ELB.8905)": The backend server group quota of the shared load balancer has been used up. You need to delete unnecessary backend server groups or replace the shared load balancer with a dedicated load balancer.
- Add the following annotations to the Kourier Service (the Kourier Service in the kourier-system namespace):
- Load balancer type selection
- Image repository planning
By default, Knative component images are obtained from gcr.io/knative-releases and ghcr.io/knative. If CCE nodes cannot directly pull the images over the public network, you need to migrate the images to SWR. For details about how to migrate images, see Pushing an Image.
Component
Default Image
SWR Target
Activator/Autoscaler/Controller/Webhook
gcr.io/knative-releases/knative.dev/serving/cmd/*
swr.<region>.myhuaweicloud.com/<project>/knative-*:v1.19.6
net-kourier-controller/kourier-gateway
gcr.io/knative-releases/knative.dev/net-kourier/cmd/*
swr.<region>.myhuaweicloud.com/<project>/kourier-*:v1.19.0
Procedure
Step 1: Install kn.
kn is the command line client of Knative. It is used to create, update, and delete Knative Services and manage revisions.
- Use kubectl to connect to the cluster. For details, see Using kubectl to Access a Cluster.
- Run the command below to install kn. Knative v1.19.0 is used as an example.
# Linux amd64 curl -Lo ./kn https://github.com/knative/client/releases/download/knative-v1.19.0/kn-linux-amd64 chmod +x ./kn sudo mv ./kn /usr/local/bin/
- Verify the installation.
kn version
Expected output:
Version: v1.19.0 Build Date: 2026-xx-xx Git Revision: xxxxxxx Supported APIs: * Serving v1 (knative-serving v1.19.x)
Step 2: Install the CRDs and core components of Knative Serving.
If nodes cannot pull images over the public network, migrate the images to SWR and replace image path in the following YAML file with the SWR address:
- Install the CRDs.
kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.19.6/serving-crds.yaml kubectl wait --for=condition=Established --all crd --timeout=120s
The expected output contains the following information:
customresourcedefinition.apiextensions.k8s.io/services.serving.knative.dev created customresourcedefinition.apiextensions.k8s.io/configurations.serving.knative.dev created customresourcedefinition.apiextensions.k8s.io/revisions.serving.knative.dev created customresourcedefinition.apiextensions.k8s.io/routes.serving.knative.dev created ...
- Install the core components.
kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.19.6/serving-core.yaml
The components are deployed in the knative-serving namespace using the YAML file.
Component
Description
Controller
Listens to changes of Service/Configuration CRDs and coordinates the generation of revisions and Deployments.
Webhook
Verifies the CRD validity and injects default values.
Autoscaler
Collects pod concurrency metrics, adjusts the number of replicas in the Deployments, and supports scaling down to 0.
Activator
Intercepts requests when the number of pods is scaled down to 0, triggers scale-out, and forwards the requests in a queue.
- Run the following command to check whether all pods in the knative-serving namespace are in the Running state:
kubectl get pods -n knative-serving
Expected output:
NAME READY STATUS RESTARTS AGE activator-xxxxx-xxxxx 1/1 Running 0 60s autoscaler-xxxxx-xxxxx 1/1 Running 0 60s controller-xxxxx-xxxxx 1/1 Running 0 60s webhook-xxxxx-xxxxx 1/1 Running 0 60s
If the Webhook is in the CrashLoopBackOff state, it is mostly because the TLS certificate is not ready. Wait for 1 to 2 minutes until the Webhook automatically recovers. Knative v1.19 has built-in certificates. You do not need to deploy cert-manager.
Step 3: Deploy the Kourier and Configure an Ingress
Knative supports multiple types of ingresses (Kourier, Istio, and Contour). Kourier is a lightweight ingress based on Envoy. It has few dependencies and can be started quickly. It is recommended as the default gateway for beginners and production.
- Deploy Kourier.
kubectl apply -f https://github.com/knative/net-kourier/releases/download/knative-v1.19.0/kourier.yaml
Kourier is deployed in the kourier-system namespace and contains 3scale-kourier-gateway (Envoy on the data plane). The ingress is exposed through the LoadBalancer Service. CCE does not automatically associate the load balancer with the Kourier Service and allocate an external IP address. You need to manually associate the created load balancer with the Kourier Service using an annotation. If the ELB annotation is not configured, hws-cloudprovider reports the error message "listener is empty" or "elb.id is not defined, skip".
- Configure Kourier as the default ingress.
kubectl patch configmap/config-network \ --namespace knative-serving \ --type merge \ --patch '{"data":{"ingress-class":"kourier.ingress.networking.knative.dev"}}' - Verify that the ingress is ready and obtain the access IP address.
kubectl get pods -n kourier-system kubectl get svc kourier -n kourier-system
Expected output:
NAME READY STATUS RESTARTS AGE 3scale-kourier-control-xxxxx 1/1 Running 0 60s 3scale-kourier-gateway-xxxxx 1/1 Running 0 60s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kourier LoadBalancer 10.x.x.x 121.xxx.xxx.xxx 80:3xx/TCP,443:3xx/TCP 90s
EXTERNAL-IP is the public or private IP address bound to the load balancer. You need to associate the load balancer in advance using the kubernetes.io/elb.id annotation. Record this IP address, which will be used for domain name resolution.
Step 4: Configure the domain name and DNS.
The default domain name suffix of Knative is determined by the config-domain ConfigMap. The access address of all Knative Services is <service-name>.<namespace>.<domain>.
- Test environment: Use sslip.io for automatic resolution.
sslip.io resolves {ip}.sslip.io to {ip}. You do not need to set up DNS. Use the load balancer's IP address obtained in the previous step to combine a domain name.
INGRESS_IP=$(kubectl get svc kourier -n kourier-system -o jsonpath='{.status.loadBalancer.ingress[0].ip}') KNATIVE_DOMAIN="${INGRESS_IP}.sslip.io" kubectl patch configmap/config-domain \ --namespace knative-serving \ --type merge \ --patch "{\"data\":{\"${KNATIVE_DOMAIN}\":\"\"}}" echo "Knative domain: ${KNATIVE_DOMAIN}"The expected output is as follows:
Knative domain: 121.xxx.xxx.xxx.sslip.io
- Production environment: Use a custom domain name.
- Resolve the wildcard domain name *.knative.<your-domain> to the EIP bound to the load balancer.
- Configure config-domain.
kubectl patch configmap/config-domain \ --namespace knative-serving \ --type merge \ --patch '{"data":{"knative.example.com":""}}' - (Optional) Configure a TLS certificate for port 443 of the Kourier. You are advised to use CCE certificate management or secret injection for HTTPS access.
Step 5: Deploy the first Knative Service.
- Use Knative to deploy helloworld.
kn service create helloworld \ --image ghcr.io/knative/helloworld-go:latest \ --port 8080 \ --env TARGET=World
If the nodes cannot pull images from ghcr.io, migrate the images to SWR and replace them with swr.<region>.myhuaweicloud.com/<project>/helloworld-go:latest.
Expected output:
Creating service 'helloworld' in namespace 'default': 0.030s The Route is still working to reflect the latest desired specification. 0.062s ... 0.080s Configuration "helloworld" is waiting for a Revision to become ready. 8.156s ... 8.178s Ingress has not yet been reconciled. 8.298s Ready to serve. Service 'helloworld' created with latest revision 'helloworld-00001' and URL: http://helloworld.default.121.xxx.xxx.xxx.sslip.io
- View the Service and underlying resources.
kn service list kubectl get configuration,revision,route,deployment -l serving.knative.dev/service=helloworld
Expected output:
NAME URL LATEST CREATED LATEST READY READY REASON helloworld http://helloworld.default.121.xxx.xxx.xxx.sslip.io helloworld-00001 helloworld-00001 True NAME READY STATUS configuration.serving.knative.dev/helloworld True revision.serving.knative.dev/helloworld-00001 True route.serving.knative.dev/helloworld True deployment.apps/helloworld-00001-deployment 1/1
According to the auto scaling mechanism of Knative Serving, when there is no incoming traffic, the number of Deployment replicas corresponding to the revision will be automatically reduced to 0 (that is, 0/0).
Knative Service automatically creates four levels of resources: Configuration, Revision, Route, and Deployment. The CRD hierarchy is as follows: Service → Configuration → Revision → Deployment.
- Access the Service. Test the helloworld Service on any node or machine that can access the public network.
curl http://helloworld.default.${KNATIVE_DOMAIN}If "Hello World!" is returned, the Service is successfully deployed.
Step 6: Verify scaling to zero.
Scale to zero is a core feature of Knative. When there is no traffic, the Autoscaler reduces the number of Deployment replicas to 0. When a request arrives, the Activator intercepts the request and triggers scale-out.
- Observe the scale-down to zero.
After the Service has no requests for a period of time (the default idle window is 30 seconds), the pod is automatically terminated.
# Continuously observe the pod status. kubectl get pod -l serving.knative.dev/service=helloworld -w
Expected result: After about 30 to 60 seconds, the number of pods changes to 0.
NAME READY STATUS RESTARTS AGE helloworld-00001-deployment-xxxxx-xxxxx 2/2 Running 0 60s helloworld-00001-deployment-xxxxx-xxxxx 2/1 Terminating 0 90s helloworld-00001-deployment-xxxxx-xxxxx 0/1 Terminating 0 95s
- Trigger a cold start.
When the number of pods is 0 and a request is initiated, the Activator intercepts the request, triggers scale-out, and forwards the request.
curl -w "\nTotal time: %{time_total}s\n" http://helloworld.default.${KNATIVE_DOMAIN}Expected output:
Hello World! Total time: 1.2s
The duration of the first cold start depends on the image pull and application startup duration (usually 1 to 3 seconds). Subsequent requests will directly hit the running pod, reducing the time to milliseconds.
- (Optional) Adjust the scale-down window.
By default, Knative reduces the number of pods to 0 after the pods are idle for 30 seconds. You can adjust the window by using the revision annotation, for example, extending the window to 1 minute.
kn service update helloworld \ --annotation autoscaling.knative.dev/scale-to-zero-pod-retention-period=1m
Expected output:
Updating service 'helloworld' in namespace 'default': 0.030s The Route is still working to reflect the latest desired specification. 0.062s ... 0.080s Configuration "helloworld" is waiting for a Revision to become ready. 1.156s ... 1.178s Ingress has not yet been reconciled. 1.298s Ready to serve. Service 'helloworld' updated to latest revision 'helloworld-00003' and URL: http://helloworld.default.121.xxx.xxx.xxx.sslip.io
Modifying the annotation will trigger Knative to create a new revision (for example, helloworld-00003 in the example). The old revision will gradually stop receiving traffic, and the new revision will handle subsequent requests.
Step 7: Verify the grayscale release (blue-green deployment or canary release).
Knative implements grayscale releases using immutable revisions and traffic weights, without the need to introduce additional toolchains.
- Update the image. (A new revision is automatically generated.)
kn service update helloworld \ --image ghcr.io/knative/helloworld-go:latest \ --env TARGET=Knative
This operation will automatically create a new revision (helloworld-00002), and the default route will switch 100% traffic to the new version. The old revision (helloworld-00001) is retained but does not receive traffic.
- Canary release: Traffic is distributed based on weights.
The traffic is split: 90% to the old version and 10% to the new version. After the observation, the traffic is gradually switched.
kn service update helloworld \ --traffic helloworld-00001=90 \ --traffic helloworld-00002=10
The --traffic parameter is in the format of Revision name=Percentage. The total weight of multiple revisions must be 100.
- Verify the traffic distribution. Request the Service multiple times and observe the distribution of returned content.
for i in $(seq 1 10); do curl -s http://helloworld.default.${KNATIVE_DOMAIN} done | sort | uniq -cExpected output (about 9:1):
9 Hello World! 1 Hello Knative!
- View route details.
kn route describe helloworld
Expected output:
Name: helloworld Namespace: default Age: 10m URL: http://helloworld.default.121.xxx.xxx.xxx.sslip.io Traffic Targets: 90% @latest (helloworld-00001) # still old URL: http://helloworld.default.../helloworld-00001 10% helloworld-00002 URL: http://helloworld.default.../helloworld-00002 - Roll back the version.
A revision is an immutable snapshot. The rollback operation redirects the traffic to the old revision based on the weights.
kn service update helloworld --traffic helloworld-00001=100
After the command is executed, the traffic is immediately switched back to helloworld-00001, and no redeployment is required.
Configuration Optimization
KPA Auto Scaling Optimization
By default, KPA performs auto scaling based on the number of concurrent requests. In the production environment, you need to adjust key parameters based on service characteristics.
Key parameters are configured using template.metadata.annotations of the revision.
| Annotation | Default Value | Description | Suggestions |
|---|---|---|---|
autoscaling.knative.dev/metric | concurrency | Scaling metric | Set the value to rps for short tasks with high QPS and to concurrency for persistent connection or recalculation tasks. |
autoscaling.knative.dev/target | 100 (concurrency)/200 (rps) | Target concurrency/Rate per pod | Set this parameter based on the single-pod bearing capability. If the value is too low, excessive scale-out will be performed. If the value is too high, traffic limiting will be triggered. |
autoscaling.knative.dev/max-scale | 0 (no upper limit) | Maximum number of pods | This parameter is mandatory in the production environment to prevent the node pool resources from being used up due to sudden traffic increase. |
autoscaling.knative.dev/min-scale | 0 | Minimum number of pods | Set this parameter to 1 or 2 for latency-sensitive workloads to reserve pods to be kept warm. |
autoscaling.knative.dev/scale-to-zero-pod-retention-period | 0s | Retention duration before scaling down to 0 | Set this parameter to 30s to 1 minute to smoothly handle intermittent traffic. |
autoscaling.knative.dev/window | 60s | Scaling calculation window | In burst traffic scenarios, the value is shortened to 30s. In stable traffic scenarios, the value is prolonged to 120s to reduce jitter. |
The annotations in this table apply to a single revision. To set global default values for all Services, edit config-autoscaler ConfigMap: kubectl edit configmap/config-autoscaler -n knative-serving. The annotation priority is higher than the global default values.
Example: Configuring KPA for a high-QPS web service
The following uses the helloworld Service as an example to describe how to configure KPA parameters for high-QPS scenarios:
kn service create web-api \ --image swr.cn-north-7.myhuaweicloud.com/<project>/web-api:latest \ --port 8080 \ --annotation autoscaling.knative.dev/metric=rps \ --annotation autoscaling.knative.dev/target=200 \ --annotation autoscaling.knative.dev/max-scale=20 \ --annotation autoscaling.knative.dev/min-scale=2 \ --annotation autoscaling.knative.dev/window=30s
Resource Quotas
To prevent the control plane from being overloaded, pod scheduling failures, or cluster resources from being exhausted, you are advised to configure resource quotas based on the policies below.
| Configuration Item | Recommended Value | Description |
|---|---|---|
| Control plane pod resource requests | Controller/Autoscaler: 500m/512Mi; Webhook 100m/50Mi | The default values are low. You are advised to increase the values for production workloads. |
| Service pod requests | Set the values based on the bearer capability of a single pod. | If the values are too low, node resources may be overcommitted. If the values are too high, scheduling may fail. By default, Knative calculates the number of concurrent requests based on this configuration item. |
| Service pod limits | 2 to 4 times of service pod requests | Reserve resources for traffic bursts to prevent OOM caused by traffic bursts. In the GPU scenario, the limits must be equal to requests. |
| namespace ResourceQuota | Set the values based on the requirements of the service namespace. | Limit the resource quotas of a single namespace to prevent cluster resources from being exhausted or services from being unavailable due to unlimited scale-out of Knative Services. |
Adjust the default resource policy globally through the config-resource ConfigMap:
kubectl patch configmap/config-resource \
--namespace knative-serving \
--type merge \
--patch '{"data":{"pod-container-default-requests":"{\"cpu\":\"500m\",\"memory\":\"512Mi\"}"}}' Common Issues
What is your overall rating for this page?
Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot