NGNIX Ingress Controller

Introduction

Kubernetes uses kube-proxy to expose Services and provide load balancing. The implementation is at the transport layer. When it comes to Internet applications, where a bucket-load of information is generated, forwarding needs to be more fine-grained, precisely and flexibly controlled by policies and load balancers to deliver higher performance.

This is where ingresses enter. Ingresses provide application-layer forwarding functions, such as virtual hosts, load balancing, SSL proxy, and HTTP routing, for Services that can be directly accessed outside a cluster.

Kubernetes has officially released the Nginx-based Ingress controller. CCE nginx-ingress directly uses community templates and images. The Nginx Ingress controller generates Nginx configuration and stores the configuration using ConfigMap. The configuration will be written to Nginx pods through the Kubernetes API. In this way, the Nginx configuration is modified and updated. For details, see How nginx-ingress Works.

You can visit the open source community for more information.

When installing the add-on, you can add configurations by defining the Nginx configuration. The configurations take effect globally. This parameter is generated by configuring the nginx.conf file and affects all managed ingresses. You can search for related parameters in the ConfigMap. If the configured parameters are not included in the options listed in the ConfigMap, the configurations do not take effect.
Do not manually modify or delete the load balancer and listener that are automatically created by CCE. Otherwise, the workload will be abnormal. If you have modified or deleted them by mistake, uninstall the nginx-ingress add-on and re-install it.

How nginx-ingress Works

nginx-ingress consists of the ingress object, ingress controller, and Nginx. The ingress controller assembles ingresses into the Nginx configuration file (nginx.conf) and reloads Nginx to make the changed configurations take effect. When it detects that the pod in a Service changes, it dynamically changes the upstream server group configuration of Nginx. In this case, the Nginx process does not need to be reloaded. Figure 1 shows how nginx-ingress works.

An ingress is a group of access rules that forward requests to specified Services based on domain names or URLs. Ingresses are stored in the object storage service etcd and are added, deleted, modified, and queried through APIs.
The ingress controller monitors the changes of resource objects such as ingresses, Services, endpoints, secrets (mainly TLS certificates and keys), nodes, and ConfigMaps in real time and automatically performs operations on Nginx.
Nginx implements load balancing and access control at the application layer.

Figure 1 Working principles of nginx-ingress

Constraints

This add-on can be installed only in CCE clusters of v1.15 or later.
kubernetes.io/ingress.class: "nginx" must be added to the annotation of the Ingress created through the API.
Dedicated load balancers must be the network type (TCP/UDP) supporting private networks (with a private IP).
The node where nginx-ingress-controller is running and the containers running on the node cannot access Nginx Ingress. In this case, perform anti-affinity deployment for the workloads and nginx-ingress-controller. For details, see Anti-affinity Deployment for Workloads and nginx-ingress-controller.

Prerequisites

Before creating a workload, you must have an available cluster. If no cluster is available, create one according to Creating a Cluster.

Installing the Add-on

The CVE-2021-25746 vulnerability has been fixed in nginx-ingress-controller of v1.2.0 (corresponding to CCE nginx-ingress add-on 2.1.0). Rules are added to disable some annotations prone to unauthorized access.
The CVE-2021-25745 vulnerability has been fixed in nginx-ingress-controller of v1.2.0 (corresponding to CCE nginx-ingress add-on 2.1.0). Rules are added to disable some access paths prone to unauthorized access.

Log in to the CCE console and click the cluster name to access the cluster console. Choose Add-ons in the navigation pane, locate NGNIX Ingress Controller on the right, and click Install.

On the Install Add-on page, configure the specifications.

**Table 1** Add-on configuration
Parameter	Description
Add-on Specifications	The specifications can be Custom.
Pods	If you select Custom, you can adjust the number of pods as required.
Multi-AZ	Preferred: Deployment pods of the add-on will be preferentially scheduled to nodes in different AZs. If all the nodes in the cluster are deployed in the same AZ, the pods will be scheduled to that AZ. Required: Deployment pods of the add-on will be forcibly scheduled to nodes in different AZs. If there are fewer AZs than pods, the extra pods will fail to run.
Containers	If you select Custom, you can adjust the container specifications as required.

Configure the add-on parameters.
- Load Balancer: Select a shared or dedicated load balancer. If no load balancer is available, create one first. The load balancer has at least two listeners, and ports 80 and 443 are not occupied by listeners.
- Nginx Parameters: Configuring the nginx.conf file will affect all managed ingresses. You can search for related parameters through ConfigMaps. If the parameters you configured are not included in the options listed in those ConfigMaps, the parameters will not take effect.
  For example, you can use the keep-alive-requests parameter to describe how to set the maximum number of requests for keeping active connections to 100.
```
{
    "keep-alive-requests": "100"
}
```
- Default 404 Service: By default, the 404 service provided by the add-on is used. To customize the 404 service, enter the namespace/service name. If the service does not exist, the add-on installation will fail.
Click Install.

Components

**Table 2** nginx-ingress component
Container Component	Description	Resource Type
nginx-ingress	Nginx Ingress controller, which provides flexible routing and forwarding for clusters	Deployment

Anti-affinity Deployment for Workloads and nginx-ingress-controller

The node where nginx-ingress-controller is running and the containers running on the node cannot access Nginx Ingress. To prevent this problem, configure an anti-affinity rule to tell the scheduler not to co-locate the workload and nginx-ingress-controller on the same node.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx:aplpine
        imagePullPolicy: IfNotPresent
        name: nginx
      imagePullSecrets:
      - name: default-secret
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:    # Use the labels of nginx-ingress-controller to implement anti-affinity.
                  - key: app
                    operator: In
                    values:
                      - nginx-ingress
                  - key: component
                    operator: In
                    values:
                      - controller
              namespaces:
                - kube-system
              topologyKey: kubernetes.io/hostname