Advanced Configuration of Nginx Ingress Controller

Parameter Optimization in High-Concurrency Scenarios

In high-concurrency scenarios, you can configure parameters for optimization in either of the following ways:

1. Use ConfigMap to optimize the overall parameters of Nginx Ingress Controller.
2. Use InitContainers to optimize the kernel parameters of Nginx Ingress Controller.

The optimized value.yaml configuration file is as follows:

controller:
  image:
    repository: registry.k8s.io/ingress-nginx/controller
    registry: ""
    image: ""
    tag: "v1.5.1"  # Controller version
    digest: ""
  ingressClassResource:
    name: ccedemo         # The name of each Ingress Controller in the same cluster must be unique and cannot be nginx or cce.
    controllerValue: "k8s.io/ingress-nginx-demo"  # The listening identifier of each Ingress Controller in the same cluster must be unique and cannot be set to k8s.io/ingress-nginx.
  ingressClass: ccedemo   # The name of each Ingress Controller in the same cluster must be unique and cannot be nginx or cce.
  service: 
    annotations: 
      kubernetes.io/elb.id: 5083f225-9bf8-48fa-9c8b-67bd9693c4c0     #ELB ID
    kubernetes.io/elb.class: performance  # This annotation is required only for dedicated load balancers.
# Nginx parameter optimization
  config: 
    keep-alive-requests: 10000
    upstream-keepalive-connections: 200
    max-worker-connections: 65536
# Kernel parameter optimization
  extraInitContainers: 
   - name: init-myservice
     image: busybox
     securityContext:
       privileged: true
     command: ['sh', '-c', 'sysctl -w net.core.somaxconn=65535;sysctl -w net.ipv4.ip_local_port_range="1024 65535"']
  extraVolumeMounts: # Mount the /etc/localtime file on the node to synchronize the time zone.
    - name: localtime
      mountPath: /etc/localtime
      readOnly: true
  extraVolumes:
    - name: localtime
      type: Hostpath
      hostPath:
        path: /etc/localtime 
  admissionWebhooks: # Disable webhook authentication.
    enabled: false
    patch:
      enabled: false
  resources: # Set the controller's resource limit, which can be customized.
    requests:
      cpu: 200m
      memory: 200Mi
defaultBackend: # Set defaultBackend.
  enabled: true
  image: 
    repository: registry.k8s.io/defaultbackend-amd64
    registry: ""
    image: ""
    tag: "1.5"
    digest: ""

admissionWebhook Configuration

Nginx Ingress Controller supports admissionWebhook configuration. You can configure the controller.admissionWebhook parameter to verify the validity of ingress objects. This prevents ingress-controller from continuously reloading resources due to incorrect configuration, which may cause service interruption.

• When the admissionWebhook feature is used, webhook-related configurations must be enabled on the API server, including MutatingAdmissionWebhook and ValidatingAdmissionWebhook.

  The feature switch is --admission-control=MutatingAdmissionWebhook,ValidatingAdmissionWebhook.

  If it is not enabled, submit a service ticket to enable it.

• After admissionWebhook is enabled, if you need to uninstall and reinstall Nginx Ingress Controller, residual secrets exist and need to be manually cleared.

The value.yaml configuration file for enabling admissionWebhook is as follows:

controller:
  image:
    repository: registry.k8s.io/ingress-nginx/controller
    registry: ""
    image: ""
    tag: "v1.5.1"  # Controller version
    digest: ""
  ingressClassResource:
    name: ccedemo         # The name of each Ingress Controller in the same cluster must be unique and cannot be nginx or cce.
    controllerValue: "k8s.io/ingress-nginx-demo"  # The listening identifier of each Ingress Controller in the same cluster must be unique and cannot be set to k8s.io/ingress-nginx.
  ingressClass: ccedemo   # The name of each Ingress Controller in the same cluster must be unique and cannot be nginx or cce.
  service: 
    annotations: 
      kubernetes.io/elb.id: 5083f225-9bf8-48fa-9c8b-67bd9693c4c0     #ELB ID
    kubernetes.io/elb.class: performance  # This annotation is required only for dedicated load balancers.
  config: 
    keep-alive-requests: 100
  extraVolumeMounts: # Mount the /etc/localtime file on the node to synchronize the time zone.
    - name: localtime
      mountPath: /etc/localtime
      readOnly: true
  extraVolumes:
    - name: localtime
      type: Hostpath
      hostPath:
        path: /etc/localtime 
  admissionWebhooks:
    annotations: {}
    enabled: true
    extraEnvs: []
    failurePolicy: Fail
    port: 8443
    certificate: "/usr/local/certificates/cert"
    key: "/usr/local/certificates/key"
    namespaceSelector: {}
    objectSelector: {}
    labels: {}
    existingPsp: ""
    networkPolicyEnabled: false
    service:
      annotations: {}
      externalIPs: []
      loadBalancerSourceRanges: []
      servicePort: 443
      type: ClusterIP
    createSecretJob:
      resources:           #Annotation{}
        limits:
          cpu: 20m
          memory: 40Mi
        requests:
          cpu: 10m
          memory: 20Mi
    patchWebhookJob:
      resources: {}
    patch:
      enabled: true
      image:
        registry: registry.k8s.io      #registry.k8s.io is the image repository of the webhook official website. Replace it with the address of the repository where the image is located.
        image: ingress-nginx/kube-webhook-certgen         # webhook image
        tag: v1.1.1     
        digest: ""
        pullPolicy: IfNotPresent
      priorityClassName: ""
      podAnnotations: {}
      nodeSelector:
        kubernetes.io/os: linux
      tolerations: []
      labels: {}
      securityContext:
        runAsNonRoot: true
        runAsUser: 2000
        fsGroup: 2000
  resources: # Set the controller's resource limit, which can be customized.
    requests:
      cpu: 200m
      memory: 200Mi
defaultBackend: # Set defaultBackend.
  enabled: true
  image: 
    repository: registry.k8s.io/defaultbackend-amd64
    registry: ""
    image: ""
    tag: "1.5"
    digest: ""

Check whether admissionWebhook is verified when incorrect annotations are configured for the ingress.

For example, configure the following incorrect annotations for the ingress:

...
  annotations:
    nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "false"
    nginx.ingress.kubernetes.io/auth-tls-verify-client: optional
    nginx.ingress.kubernetes.io/auth-tls-verify-depth: "1"
...

When the ingress service is created, the following interception information is displayed: