Nginx Ingress Controller高级配置

高并发业务场景参数优化

针对高并发业务场景，可通过参数配置进行优化：

通过ConfigMap对Nginx Ingress Controller整体参数进行优化。
通过InitContainers对Nginx Ingress Controller内核参数进行优化。

优化后的value.yaml配置文件如下：

controller:
  image:
    repository: registry.k8s.io/ingress-nginx/controller
    registry: ""
    image: ""
    tag: "v1.5.1"  #controller版本
    digest: ""
  ingressClassResource:
    name: ccedemo         #同一个集群中不同套Ingress Controller名称必须唯一，且不能设置为nginx和cce
    controllerValue: "k8s.io/ingress-nginx-demo"  #同一个集群中不同套Ingress Controller的监听标识必须唯一，且不能设置为k8s.io/ingress-nginx
  ingressClass: ccedemo   #同一个集群中不同套Ingress Controller名称必须唯一，且不能设置为nginx和cce
  service: 
    annotations: 
      kubernetes.io/elb.id: 5083f225-9bf8-48fa-9c8b-67bd9693c4c0     #ELB ID
      kubernetes.io/elb.class: performance  #仅独享型ELB需要添加此注解
# Nginx参数优化
  config: 
    keep-alive-requests: 10000
    upstream-keepalive-connections: 200
    max-worker-connections: 65536
# 内核参数优化
  extraInitContainers: 
   - name: init-myservice
     image: busybox
     securityContext:
       privileged: true
     command: ['sh', '-c', 'sysctl -w net.core.somaxconn=65535;sysctl -w net.ipv4.ip_local_port_range="1024 65535"']
  extraVolumeMounts: # 挂载节点上的/etc/localtime文件，进行时区同步
    - name: localtime
      mountPath: /etc/localtime
      readOnly: true
  extraVolumes:
    - name: localtime
      type: Hostpath
      hostPath:
        path: /etc/localtime 
  admissionWebhooks: # 关闭webhook验证开关
    enabled: false
    patch:
      enabled: false
  resources: # 设定controller的资源限制，可根据需求自定义
    requests:
      cpu: 200m
      memory: 200Mi
defaultBackend: # 设置defaultBackend
  enabled: true
  image: 
    repository: registry.k8s.io/defaultbackend-amd64
    registry: ""
    image: ""
    tag: "1.5"
    digest: ""

admissionWebhook配置

Nginx Ingress Controller支持admissionWebhook配置，通过设置controller.admissionWebhook参数，可以对Ingress对象进行有效性校验，避免因配置错误导致ingress-controller不断重新加载资源，导致业务中断。

使用admissionWebhook特性时，APIServer需要开启webhook相关配置，必须包含MutatingAdmissionWebhook与ValidatingAdmissionWebhook
特性开关为 --admission-control=MutatingAdmissionWebhook,ValidatingAdmissionWebhook

如未开启，需要提交工单申请开启。
开启admissionWebhook特性后，如需卸载重装Nginx Ingress Controller，会有Secret残留，需要手动清理。

开启admissionWebhook的value.yaml配置文件如下：

controller:
  image:
    repository: registry.k8s.io/ingress-nginx/controller
    registry: ""
    image: ""
    tag: "v1.5.1"  #controller版本
    digest: ""
  ingressClassResource:
    name: ccedemo         #同一个集群中不同套Ingress Controller名称必须唯一，且不能设置为nginx和cce
    controllerValue: "k8s.io/ingress-nginx-demo"  #同一个集群中不同套Ingress Controller的监听标识必须唯一，且不能设置为k8s.io/ingress-nginx
  ingressClass: ccedemo   #同一个集群中不同套Ingress Controller名称必须唯一，且不能设置为nginx和cce
  service: 
    annotations: 
      kubernetes.io/elb.id: 5083f225-9bf8-48fa-9c8b-67bd9693c4c0     #ELB ID
      kubernetes.io/elb.class: performance  #仅独享型ELB需要添加此注解
  config: 
    keep-alive-requests: 100
  extraVolumeMounts: # 挂载节点上的/etc/localtime文件，进行时区同步
    - name: localtime
      mountPath: /etc/localtime
      readOnly: true
  extraVolumes:
    - name: localtime
      type: Hostpath
      hostPath:
        path: /etc/localtime 
  admissionWebhooks:
    annotations: {}
    enabled: true
    extraEnvs: []
    failurePolicy: Fail
    port: 8443
    certificate: "/usr/local/certificates/cert"
    key: "/usr/local/certificates/key"
    namespaceSelector: {}
    objectSelector: {}
    labels: {}
    existingPsp: ""
    networkPolicyEnabled: false
    service:
      annotations: {}
      externalIPs: []
      loadBalancerSourceRanges: []
      servicePort: 443
      type: ClusterIP
    createSecretJob:
      resources:           #注释{}
        limits:
          cpu: 20m
          memory: 40Mi
        requests:
          cpu: 10m
          memory: 20Mi
    patchWebhookJob:
      resources: {}
    patch:
      enabled: true
      image:
        registry: registry.k8s.io      #registry.k8s.io为webhook官网镜像仓库，需要替换成自己镜像所在仓库地址
        image: ingress-nginx/kube-webhook-certgen         #webhook镜像
        tag: v1.1.1     
        digest: ""
        pullPolicy: IfNotPresent
      priorityClassName: ""
      podAnnotations: {}
      nodeSelector:
        kubernetes.io/os: linux
      tolerations: []
      labels: {}
      securityContext:
        runAsNonRoot: true
        runAsUser: 2000
        fsGroup: 2000
  resources: # 设定controller的资源限制，可根据需求自定义
    requests:
      cpu: 200m
      memory: 200Mi
defaultBackend: # 设置defaultBackend
  enabled: true
  image: 
    repository: registry.k8s.io/defaultbackend-amd64
    registry: ""
    image: ""
    tag: "1.5"
    digest: ""

验证Ingress配置错误annotation场景下，admissionWebhook是否会进行校验。

例如，为Ingress配置以下错误的annotation：

...
  annotations:
    nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "false"
    nginx.ingress.kubernetes.io/auth-tls-verify-client: optional
    nginx.ingress.kubernetes.io/auth-tls-verify-depth: "1"
...

创建此Ingress服务，将会出现以下拦截信息：

点击放大