Help Center/ Cloud Container Engine/ Best Practices/ Auto Scaling/ Auto Scaling of Multiple Applications Using Nginx Ingresses

Updated on 2025-01-08 GMT+08:00

View PDF

Auto Scaling of Multiple Applications Using Nginx Ingresses

Deploying applications in multiple pods in a production environment can enhance their stability and reliability, but it can also lead to increased resource waste and costs. To strike a balance between resource utilization and application performance, manually adjusting the number of pods may not be efficient or effective.

However, if the application uses Nginx ingresses to route and forward external traffic, you can configure HPA policies using the nginx_ingress_controller_requests metric. This allows for dynamic adjustment of pods based on traffic changes, optimizing resource utilization.

Prerequisites

The NGINX Ingress Controller add-on has been installed in the cluster.
You have installed the Cloud Native Cluster Monitoring add-on in the cluster and enabled Local Data Storage for the add-on.
You have connected the cluster with the kubectl command line tool or CloudShell.
The pressure testing tool Apache Benchmark has been installed.

Creating a Workload and a Service for the Workload

This section provides an example of how to route external traffic for two Services using Nginx ingresses.

Create a test-app workload and a Service for it.

Write a test-app.yaml file.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-app
  labels:
    app: test-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test-app
  template:
    metadata:
      labels:
        app: test-app
    spec:
      containers:
      - image: skto/sample-app:v2
        name: metrics-provider
        ports:
        - name: http
          containerPort: 8080
--- 
apiVersion: v1
kind: Service
metadata:
  name: test-app
  namespace: default
  labels:
    app: test-app
spec:
  ports:
    - port: 8080
      name: http
      protocol: TCP
      targetPort: 8080
  selector: 
    app: test-app 
  type: ClusterIP

Deploy the test-app workload and the corresponding Service.
```
kubectl apply -f test-app.yaml
```

Create a sample-app workload and a Service for it.

Write a sample-app.yaml file.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-app
  labels:
    app: sample-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sample-app
  template:
    metadata:
      labels:
        app: sample-app
    spec:
      containers:
      - image: skto/sample-app:v2
        name: metrics-provider
        ports:
        - name: http
          containerPort: 8080
--- 
apiVersion: v1
kind: Service
metadata:
  name: sample-app
  namespace: default
  labels:
    app: sample-app
spec:
  ports:
    - port: 80
      name: http
      protocol: TCP
      targetPort: 8080
  selector:
    app: sample-app
  type: ClusterIP

Deploy the sample-app workload and the corresponding Service.
```
kubectl apply -f sample-app.yaml
```

Deploy an ingress.
1. Write an ingress.yaml file.
```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  namespace: default
spec:
  ingressClassName: nginx
  rules:
    - host: test.example.com
      http:
        paths:
          - backend:
              service:
                name: sample-app
                port:
                  number: 80
            path: /
            pathType: ImplementationSpecific
          - backend:
              service:
                name: test-app
                port:
                  number: 8080
            path: /home
            pathType: ImplementationSpecific
```
  - host: specifies the Service access domain name. In this example, test.example.com is used.
  - path: specifies the URL to be accessed. After receiving a request, the system matches the request with the corresponding Service based on the routing rules and accesses the corresponding pod through the Service.
  - backend: consists of the Service name and Service port and specifies the Service forwarded by the current path.
2. Deploy an ingress.
```
kubectl apply -f ingress.yaml
```
3. Obtain an ingress.
```
kubectl get ingress -o wide
```
4. After the deployment is successful, log in to the target node and add the service domain name and the IP address of the load balancer associated with NGINX Ingress Controller to the local hosts file of the node. The IP address of the load balancer associated with NGINX Ingress Controller is that obtained in 3.c.
```
export NGINXELB=xx.xx.xx.xx 
echo -n "${NGINXELB}  test.example.com" >> /etc/hosts
```
5. Log in to the cluster node and access the host address through the / and /home paths.
  
  NGINX Ingress Controller accesses sample-app and test-app based on the preceding configurations.
```
# curl test.example.com/
Hello from '/' path!

# curl test.example.com/home
Hello from '/home' path!
```

Modifying user-adapter-config in Prometheus

Run the following command to edit user-adapter-config:

kubectl -n monitoring edit configmap user-adapter-config

Add the following rules to the ConfigMap of the adapter:

apiVersion: v1
data:
  config.yaml: |
    rules:
    - metricsQuery: sum(rate(<<.Series>>{<<.LabelMatchers>>}[2m])) by (<<.GroupBy>>)
      name:
        as: ${1}_per_second
        matches: ^(.*)_requests
      resources:
        namespaced: false
        overrides:
          exported_namespace:
            resource: namespace
          service:
            resource: service
      seriesQuery: nginx_ingress_controller_requests
    resourceRules:
      cpu:
        containerQuery: sum(rate(container_cpu_usage_seconds_total{<<.LabelMatchers>>,container!="",pod!=""}[1m])) by (<<.GroupBy>>)
        nodeQuery: sum(rate(container_cpu_usage_seconds_total{<<.LabelMatchers>>, id='/'}[1m])) by (<<.GroupBy>>)
        resources:
          overrides:
            instance:
              resource: node
            namespace:
              resource: namespace
            pod:
              resource: pod
        containerLabel: container
      memory:
        containerQuery: sum(container_memory_working_set_bytes{<<.LabelMatchers>>,container!="",pod!=""}) by (<<.GroupBy>>)
        nodeQuery: sum(container_memory_working_set_bytes{<<.LabelMatchers>>,id='/'}) by (<<.GroupBy>>)
        resources:
          overrides:
            instance:
              resource: node
            namespace:
              resource: namespace
            pod:
              resource: pod
        containerLabel: container

Restart custom-metrics-apiserver.

kubectl -n monitoring delete pod -l app=custom-metrics-apiserver

# curl test.example.com/
Hello from '/' path!

# curl test.example.com/home
Hello from '/home' path!

Run the following command to check whether the metric is added:

kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1/namespaces/default/services/*/nginx_ingress_controller_per_second | python -m json.tool

Click to enlarge

Creating an HPA Policy

Create an hpa.yaml file and configure auto scaling for the test-app and sample-app workloads based on Prometheus metrics.

apiVersion: autoscaling/v2 
kind: HorizontalPodAutoscaler 
metadata: 
  name: sample-hpa  # HPA name
spec: 
  scaleTargetRef: 
    apiVersion: apps/v1 
    kind: Deployment 
    name: sample-app  # Deployment name
  minReplicas: 1      # Minimum number of pods
  maxReplicas: 10     # Maximum number of pods
  metrics: 
    - type: Object 
      object: 
        metric: 
          name: nginx_ingress_controller_per_second  # Metric
        describedObject: 
          apiVersion: v1 
          kind: service 
          name: sample-app  # Service of the Deployment
        target: 
          type: Value 
          value: 30   # Scaling is triggered when the metric value is within the range of (Actual value/30)±0.1.
--- 
apiVersion: autoscaling/v2 
kind: HorizontalPodAutoscaler 
metadata: 
  name: test-hpa 
spec: 
  scaleTargetRef: 
    apiVersion: apps/v1 
    kind: Deployment 
    name: test-app 
  minReplicas: 1 
  maxReplicas: 10 
  metrics: 
    - type: Object 
      object: 
        metric: 
          name: nginx_ingress_controller_per_second 
        describedObject: 
          apiVersion: v1 
          kind: service 
          name: test-app 
        target: 
          type: Value 
          value: 30

Deploy the HPA policy.
```
kubectl apply -f hpa.yaml
```
Check the HPA deployment.
```
kubectl get hpa
```

Verifying Scaling

Log in to the target cluster node and perform a pressure testing on the /home path.
```
ab -c 50 -n 5000 test.example.com/home
```
Check the HPA.
```
kubectl get hpa
```
Log in to the target cluster node and perform a pressure testing on the root path.
```
ab -c 50 -n 5000 test.example.com/
```
Check the HPA.
```
kubectl get hpa
```
Compared with the HPA metrics obtained before the pressure testing, the service application is scaled out when the number of requests exceeds the threshold.