¿Por qué no se puede utilizar la dirección ELB para acceder a las cargas de trabajo en un clúster?
Síntoma
En un clúster (en un nodo o en un contenedor), la dirección ELB no se puede utilizar para acceder a cargas de trabajo.
Causa posible
If the service affinity of a LoadBalancer Service is set to the node level, that is, the value of externalTrafficPolicy is Local, the ELB address may fail to be accessed from the cluster (specifically, nodes or containers).
This is because when the LoadBalancer service is created, kube-proxy adds the ELB access address (external-ip) to iptables or IPVS. When the ELB address is accessed from the cluster, the ELB load balancer is not used. Instead, kube-proxy directly forwards the access request. The case depends on which container network model and service forwarding mode you use.
When externalTrafficPolicy is set to Local, the access may fail in the following scenarios:
|
Server |
Client |
Tunnel Network Cluster (IPVS) |
VPC Network Cluster (IPVS) |
Tunnel Network Cluster (iptables) |
VPC Network Cluster (iptables) |
|
NodePort Service |
Same node |
OK. The node where the pod runs is accessible, not any other nodes. |
OK. The node where the pod runs is accessible. |
OK. The node where the pod runs is accessible. |
OK. The node where the pod runs is accessible. |
|
Cross-node |
OK. The node where the pod runs is accessible, not any other nodes. |
OK. The node where the pod runs is accessible. |
OK. The node where the pod runs is accessible by visiting the node IP + port, not by any other ways. |
OK. The node where the pod runs is accessible by visiting the node IP + port, not by any other ways. |
|
|
Containers on the same node |
OK. The node where the pod runs is accessible, not any other nodes. |
OK. The node where the pod runs is not accessible. |
OK. The node where the pod runs is accessible. |
OK. The node where the pod runs is not accessible. |
|
|
Containers across nodes |
OK. The node where the pod runs is accessible, not any other nodes. |
OK. The node where the pod runs is accessible. |
OK. The node where the pod runs is accessible. |
OK. The node where the pod runs is accessible. |
|
|
LoadBalancer Service using a dedicated load balancer |
Same node |
Accessible for public networks, not private networks. |
Accessible for public networks, not private networks. |
Accessible for public networks, not private networks. |
Accessible for public networks, not private networks. |
|
Containers on the same node |
Accessible for public networks, not private networks. |
Accessible for public networks, not private networks. |
Accessible for public networks, not private networks. |
Accessible for public networks, not private networks. |
|
|
Local Service of the nginx-ingress add-on using a dedicated load balancer |
Same node |
Accessible for public networks, not private networks. |
Accessible for public networks, not private networks. |
Accessible for public networks, not private networks. |
Accessible for public networks, not private networks. |
|
Containers on the same node |
Accessible for public networks, not private networks. |
Accessible for public networks, not private networks. |
Accessible for public networks, not private networks. |
Accessible for public networks, not private networks. |
Solución
Se pueden utilizar los siguientes métodos para resolver este problema:
- (Recomendado) En el clúster, utilice el nombre de dominio de servicio o servicio de ClusterIP para el acceso.
- Establezca externalTrafficPolicy del servicio en Cluster es decir, la afinidad de servicio a nivel de clúster. Tenga en cuenta que esto afecta a la persistencia de la dirección de origen.
apiVersion: v1 kind: Service metadata: annotations: kubernetes.io/elb.class: union kubernetes.io/elb.autocreate: '{"type":"public","bandwidth_name":"cce-bandwidth","bandwidth_chargemode":"traffic","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","name":"james"}' labels: app: nginx name: nginx spec: externalTrafficPolicy: Cluster ports: - name: service0 port: 80 protocol: TCP targetPort: 80 selector: app: nginx type: LoadBalancer - Aprovechando la función de paso a través del Servicio, kube-proxy se omite cuando se utiliza la dirección de ELB para acceder. Primero se accede al balanceador de carga de ELB y, a continuación, a la carga de trabajo.
- Después de configurar las redes de paso a través para un balanceador de carga dedicado, no se puede acceder a contenedores en el nodo donde se ejecuta la carga de trabajo a través del Service.
- Las redes de paso a través no son compatibles con clústeres de v1.15 o anteriores.
- En el modo de red IPVS, la configuración de paso a través del Service conectado al mismo ELB debe ser la misma.
apiVersion: v1 kind: Service metadata: annotations: kubernetes.io/elb.pass-through: "true" kubernetes.io/elb.class: union kubernetes.io/elb.autocreate: '{"type":"public","bandwidth_name":"cce-bandwidth","bandwidth_chargemode":"traffic","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","name":"james"}' labels: app: nginx name: nginx spec: externalTrafficPolicy: Local ports: - name: service0 port: 80 protocol: TCP targetPort: 80 selector: app: nginx type: LoadBalancer
