Advanced Settings of an NGINX Ingress Controller

Parameter Optimization in High-Concurrency Scenarios

In high-concurrency scenarios, you can optimize the NGINX Ingress Controller using the following methods:

1. Use a ConfigMap to adjust global NGINX Ingress Controller parameters.
2. Use init containers to tune the NGINX Ingress Controller kernel parameters.

The optimized value.yaml example is as follows:

controller:
  image:
    repository: registry.k8s.io/ingress-nginx/controller
    registry: ""
    image: ""
    tag: "v1.5.1"  # Controller version
    digest: ""
  ingressClassResource:
    name: ccedemo         # The name of each NGINX Ingress Controller in the same cluster must be unique and cannot be nginx or cce.
    controllerValue: "k8s.io/ingress-nginx-demo"  # The listening identifier of each NGINX Ingress Controller in the same cluster must be unique and cannot be set to k8s.io/ingress-nginx.
  ingressClass: ccedemo   # The name of each NGINX Ingress Controller in the same cluster must be unique and cannot be nginx or cce.
  service: 
    annotations: 
      kubernetes.io/elb.id: 5083f225-9bf8-48fa-9c8b-67bd9693c4c0     # Load balancer ID
    kubernetes.io/elb.class: performance  # This annotation is required only for dedicated load balancers.
# Nginx parameter optimization
  config: 
    keep-alive-requests: 10000
    upstream-keepalive-connections: 200
    max-worker-connections: 65536
# Kernel parameter optimization
  extraInitContainers: 
   - name: init-myservice
     image: busybox
     securityContext:
       privileged: true
     command: ['sh', '-c', 'sysctl -w net.core.somaxconn=65535;sysctl -w net.ipv4.ip_local_port_range="1024 65535"']
  extraVolumeMounts: # Mount the /etc/localtime file on the node to synchronize the time zone.
    - name: localtime
      mountPath: /etc/localtime
      readOnly: true
  extraVolumes:
    - name: localtime
      type: Hostpath
      hostPath:
        path: /etc/localtime 
  admissionWebhooks: # Disable webhook authentication.
    enabled: false
    patch:
      enabled: false
  resources: # Set the controller's resource limit, which can be customized.
    requests:
      cpu: 200m
      memory: 200Mi
defaultBackend: # Set defaultBackend.
  enabled: true
  image: 
    repository: registry.k8s.io/defaultbackend-amd64
    registry: ""
    image: ""
    tag: "1.5"
    digest: ""

Admission Webhook Configuration

The NGINX Ingress Controller supports admission webhooks. By enabling controller.admissionWebhook, the controller validates ingress objects before applying them. This prevents the controller from repeatedly reloading resources due to invalid configurations, which could cause service interruption.

• When the admission webhooks are used, webhook-related configurations must be enabled on the API server, including MutatingAdmissionWebhook and ValidatingAdmissionWebhook.

  The feature switch is --admission-control=MutatingAdmissionWebhook,ValidatingAdmissionWebhook.

  If it is not enabled, submit a service ticket to enable it.

• After enabling admission webhooks, uninstalling and reinstalling the NGINX Ingress Controller may leave residual secrets. These must be manually removed.

The value.yaml example for enabling admission webhooks is as follows:

controller:
  image:
    repository: registry.k8s.io/ingress-nginx/controller
    registry: ""
    image: ""
    tag: "v1.5.1"  # Controller version
    digest: ""
  ingressClassResource:
    name: ccedemo         # The name of each NGINX Ingress Controller in the same cluster must be unique and cannot be nginx or cce.
    controllerValue: "k8s.io/ingress-nginx-demo"  # The listening identifier of each NGINX Ingress Controller in the same cluster must be unique and cannot be set to k8s.io/ingress-nginx.
  ingressClass: ccedemo   # The name of each NGINX Ingress Controller in the same cluster must be unique and cannot be nginx or cce.
  service: 
    annotations: 
      kubernetes.io/elb.id: 5083f225-9bf8-48fa-9c8b-67bd9693c4c0     # Load balancer ID
    kubernetes.io/elb.class: performance  # This annotation is required only for dedicated load balancers.
  config: 
    keep-alive-requests: 100
  extraVolumeMounts: # Mount the /etc/localtime file on the node to synchronize the time zone.
    - name: localtime
      mountPath: /etc/localtime
      readOnly: true
  extraVolumes:
    - name: localtime
      type: Hostpath
      hostPath:
        path: /etc/localtime 
  admissionWebhooks:
    annotations: {}
    enabled: true
    extraEnvs: []
    failurePolicy: Fail
    port: 8443
    certificate: "/usr/local/certificates/cert"
    key: "/usr/local/certificates/key"
    namespaceSelector: {}
    objectSelector: {}
    labels: {}
    existingPsp: ""
    networkPolicyEnabled: false
    service:
      annotations: {}
      externalIPs: []
      loadBalancerSourceRanges: []
      servicePort: 443
      type: ClusterIP
    createSecretJob:
      resources:           #Annotation{}
        limits:
          cpu: 20m
          memory: 40Mi
        requests:
          cpu: 10m
          memory: 20Mi
    patchWebhookJob:
      resources: {}
    patch:
      enabled: true
      image:
        registry: registry.k8s.io      #registry.k8s.io is the image repository of the webhook official website. Replace it with the address of the repository where the image is located.
        image: ingress-nginx/kube-webhook-certgen         # webhook image
        tag: v1.1.1     
        digest: ""
        pullPolicy: IfNotPresent
      priorityClassName: ""
      podAnnotations: {}
      nodeSelector:
        kubernetes.io/os: linux
      tolerations: []
      labels: {}
      securityContext:
        runAsNonRoot: true
        runAsUser: 2000
        fsGroup: 2000
  resources: # Set the controller's resource limit, which can be customized.
    requests:
      cpu: 200m
      memory: 200Mi
defaultBackend: # Set defaultBackend.
  enabled: true
  image: 
    repository: registry.k8s.io/defaultbackend-amd64
    registry: ""
    image: ""
    tag: "1.5"
    digest: ""

Verify whether the admission webhook correctly rejects invalid ingress annotations.

Configure an ingress with incorrect annotations, such as:

...
  annotations:
    nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "false"
    nginx.ingress.kubernetes.io/auth-tls-verify-client: optional
    nginx.ingress.kubernetes.io/auth-tls-verify-depth: "1"
...

When the ingress is created, the webhook returns an error similar to that shown below.