Scaling Multiple Applications Using Nginx Ingresses

Deploying a single application in multiple pods in a production environment can enhance its stability and reliability, but it can also lead to increased resource waste and costs. To strike a balance between resource utilization and application performance, manually adjusting the number of pods may not be efficient or effective.

However, if the application uses Nginx ingresses to route and forward external traffic, you can configure HPA policies using the nginx_ingress_controller_requests metric. This allows for dynamic adjustment of pods based on traffic changes, optimizing resource utilization.

Prerequisites

The NGINX Ingress Controller add-on has been installed in the cluster.
You have installed the Cloud Native Cluster Monitoring add-on in the cluster and enabled Local Data Storage for the add-on.
You have connected the cluster with kubectl or CloudShell.
The pressure testing tool Apache Benchmark has been installed.

Creating a Workload and a Service for the Workload

This section provides an example of how to route external traffic for two Services using Nginx ingresses.

Create a test-app workload and a Service for it.

Write a test-app.yaml file.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-app
  labels:
    app: test-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test-app
  template:
    metadata:
      labels:
        app: test-app
    spec:
      containers:
      - image: skto/sample-app:v2
        name: metrics-provider
        ports:
        - name: http
          containerPort: 8080
--- 
apiVersion: v1
kind: Service
metadata:
  name: test-app
  namespace: default
  labels:
    app: test-app
spec:
  ports:
    - port: 8080
      name: http
      protocol: TCP
      targetPort: 8080
  selector: 
    app: test-app 
  type: ClusterIP

Create the test-app workload and the Service.
```
kubectl apply -f test-app.yaml
```

Create a sample-app workload and a Service for it.

Write a sample-app.yaml file.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-app
  labels:
    app: sample-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sample-app
  template:
    metadata:
      labels:
        app: sample-app
    spec:
      containers:
      - image: skto/sample-app:v2
        name: metrics-provider
        ports:
        - name: http
          containerPort: 8080
--- 
apiVersion: v1
kind: Service
metadata:
  name: sample-app
  namespace: default
  labels:
    app: sample-app
spec:
  ports:
    - port: 80
      name: http
      protocol: TCP
      targetPort: 8080
  selector:
    app: sample-app
  type: ClusterIP

Create the sample-app workload and the Service.
```
kubectl apply -f sample-app.yaml
```

Deploy an ingress.
1. Write an ingress.yaml file.
```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  namespace: default
spec:
  ingressClassName: nginx
  rules:
    - host: test.example.com
      http:
        paths:
          - backend:
              service:
                name: sample-app
                port:
                  number: 80
            path: /
            pathType: ImplementationSpecific
          - backend:
              service:
                name: test-app
                port:
                  number: 8080
            path: /home
            pathType: ImplementationSpecific
```
  - host: specifies the Service access domain name. In this example, test.example.com is used.
  - path: specifies the URL to be accessed. After receiving a request, CCE matches the request with the Service based on the routing rules and accesses the pod through the Service.
  - backend: consists of the Service name and Service port and specifies the Service forwarded by the current path.
2. Create the ingress.
```
kubectl apply -f ingress.yaml
```
3. Obtain the ingress.
```
kubectl get ingress -o wide
```
4. After the deployment, log in to the node and add the service domain name and the IP address of the load balancer associated with NGINX Ingress Controller to the local hosts file of the node. The IP address of the load balancer associated with NGINX Ingress Controller is that obtained in 3.c.
```
export NGINXELB=xx.xx.xx.xx 
echo -n "${NGINXELB}  test.example.com" >> /etc/hosts
```
5. Log in to the node and access the host address through the / and /home paths.
  
  NGINX Ingress Controller accesses sample-app and test-app, respectively, based on the configurations.
```
# curl test.example.com/
Hello from '/' path!

# curl test.example.com/home
Hello from '/home' path!
```

Modifying Prometheus user-adapter-config

Edit user-adapter-config.

kubectl -n monitoring edit configmap user-adapter-config

Add the following rules to the ConfigMap of the adapter:

apiVersion: v1
data:
  config.yaml: |
    rules:
    - metricsQuery: sum(rate(<<.Series>>{<<.LabelMatchers>>}[2m])) by (<<.GroupBy>>)
      name:
        as: ${1}_per_second
        matches: ^(.*)_requests
      resources:
        namespaced: false
        overrides:
          exported_namespace:
            resource: namespace
          service:
            resource: service
      seriesQuery: nginx_ingress_controller_requests
    resourceRules:
      cpu:
        containerQuery: sum(rate(container_cpu_usage_seconds_total{<<.LabelMatchers>>,container!="",pod!=""}[1m])) by (<<.GroupBy>>)
        nodeQuery: sum(rate(container_cpu_usage_seconds_total{<<.LabelMatchers>>, id='/'}[1m])) by (<<.GroupBy>>)
        resources:
          overrides:
            instance:
              resource: node
            namespace:
              resource: namespace
            pod:
              resource: pod
        containerLabel: container
      memory:
        containerQuery: sum(container_memory_working_set_bytes{<<.LabelMatchers>>,container!="",pod!=""}) by (<<.GroupBy>>)
        nodeQuery: sum(container_memory_working_set_bytes{<<.LabelMatchers>>,id='/'}) by (<<.GroupBy>>)
        resources:
          overrides:
            instance:
              resource: node
            namespace:
              resource: namespace
            pod:
              resource: pod
        containerLabel: container

Restart custom-metrics-apiserver.

kubectl -n monitoring delete pod -l app=custom-metrics-apiserver

# curl test.example.com/
Hello from '/' path!

# curl test.example.com/home
Hello from '/home' path!

Check whether the metric has been added.

kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1/namespaces/default/services/*/nginx_ingress_controller_per_second | python -m json.tool

Click to enlarge

Creating an HPA Policy

Create an hpa.yaml file and configure auto scaling for the test-app and sample-app workloads based on Prometheus metrics.

apiVersion: autoscaling/v2 
kind: HorizontalPodAutoscaler 
metadata: 
  name: sample-hpa  # HPA name
spec: 
  scaleTargetRef: 
    apiVersion: apps/v1 
    kind: Deployment 
    name: sample-app  # Deployment name
  minReplicas: 1      # Minimum number of pods
  maxReplicas: 10     # Maximum number of pods
  metrics: 
    - type: Object 
      object: 
        metric: 
          name: nginx_ingress_controller_per_second  # Metric
        describedObject: 
          apiVersion: v1 
          kind: service 
          name: sample-app  # Service of the Deployment
        target: 
          type: Value 
          value: 30   # Scaling is triggered when the metric value is within the range of (Actual value/30)±0.1.
--- 
apiVersion: autoscaling/v2 
kind: HorizontalPodAutoscaler 
metadata: 
  name: test-hpa 
spec: 
  scaleTargetRef: 
    apiVersion: apps/v1 
    kind: Deployment 
    name: test-app 
  minReplicas: 1 
  maxReplicas: 10 
  metrics: 
    - type: Object 
      object: 
        metric: 
          name: nginx_ingress_controller_per_second 
        describedObject: 
          apiVersion: v1 
          kind: service 
          name: test-app 
        target: 
          type: Value 
          value: 30

Create the HPA policy.
```
kubectl apply -f hpa.yaml
```
Check the HPA deployment.
```
kubectl get hpa
```

Performing a Pressure Test

Log in to the node and perform a pressure test on the /home path.
```
ab -c 50 -n 5000 test.example.com/home
```
Check the HPA.
```
kubectl get hpa
```
Log in to the node and perform a pressure test on the root path.
```
ab -c 50 -n 5000 test.example.com/
```
Check the HPA.
```
kubectl get hpa
```
Compared with the HPA metrics obtained before the pressure testing, the application is scaled out when the number of requests exceeds the threshold.

Parent Topic: Auto Scaling

Previous topic: Scaling Based on Load Balancer Monitoring Metrics

Next topic: Using Karpenter for Auto Scaling of Nodes

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

For any further questions, feel free to contact us through the chatbot.

Chatbot