Symptom

In a cluster (on a node or in a container), the ELB address cannot be used to access workloads.

Possible Cause

If the service affinity of a LoadBalancer Service is set to the node level, that is, the value of externalTrafficPolicy is Local, the ELB address may fail to be accessed from the cluster (specifically, nodes or containers).

This is because when the LoadBalancer service is created, kube-proxy adds the ELB access address (external-ip) to iptables or IPVS. When the ELB address is accessed from the cluster, the ELB load balancer is not used. Instead, kube-proxy directly forwards the access request. The case depends on which container network model and service forwarding mode you use.

Solution

The following methods can be used to solve this problem:

• (Recommended) In the cluster, use the ClusterIP Service or service domain name for access.
• Set externalTrafficPolicy of the Service to Cluster, which means cluster-level service affinity. Note that this affects source address persistence.

  apiVersion: v1 
  kind: Service
  metadata: 
    annotations:   
      kubernetes.io/elb.class: union
      kubernetes.io/session-affinity-mode: SOURCE_IP
      kubernetes.io/elb.subnet-id: a9cf6d24-ad43-4f75-94d1-4e0e0464afac
      kubernetes.io/elb.autocreate: '{"type":"public","bandwidth_name":"cce-bandwidth","bandwidth_chargemode":"bandwidth","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","name":"james"}'
    labels: 
      app: nginx 
    name: nginx 
  spec: 
    externalTrafficPolicy: Cluster
    ports: 
    - name: service0 
      port: 80
      protocol: TCP 
      targetPort: 80
    selector: 
      app: nginx 
    type: LoadBalancer

• Leveraging the pass-through feature of the Service, kube-proxy is bypassed when the ELB address is used for access. The ELB load balancer is accessed first, and then the workload. For details, see Enabling Passthrough Networking for LoadBalancer Services.

  apiVersion: v1 
  kind: Service 
  metadata: 
    annotations:   
      kubernetes.io/elb.pass-through: "true"
      kubernetes.io/elb.class: union
      kubernetes.io/session-affinity-mode: SOURCE_IP
      kubernetes.io/elb.subnet-id: a9cf6d24-ad43-4f75-94d1-4e0e0464afac
      kubernetes.io/elb.autocreate: '{"type":"public","bandwidth_name":"cce-bandwidth","bandwidth_chargemode":"bandwidth","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","name":"james"}'
    labels: 
      app: nginx 
    name: nginx 
  spec: 
    externalTrafficPolicy: Local
    ports: 
    - name: service0 
      port: 80
      protocol: TCP 
      targetPort: 80
    selector: 
      app: nginx 
    type: LoadBalancer