Help Center/ Cloud Container Engine/ User Guide/ Networking/ Services/ LoadBalancer/ Configuring Advanced Load Balancing Functions Using Annotations
Updated on 2026-02-05 GMT+08:00
View PDF
Share

Configuring Advanced Load Balancing Functions Using Annotations

You can add annotations to a YAML file to use some advanced CCE functions. This section describes the available annotations when a LoadBalancer Service is created.

Indexes

Category

ELB Annotation

Load balancer configuration

Port or protocol configuration

Advanced features of ELB listeners

Interconnection with ELB

Table 1 Annotations for interconnecting with ELB

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.class

String

Select a proper load balancer type. This parameter cannot be modified after the Service is created.

Options:

The default value is union.

v1.9 or later

kubernetes.io/elb.id

String

Mandatory when an existing load balancer is associated. This parameter cannot be modified after the Service is created.

ID of a load balancer.

How to obtain:

Log in to the ELB console and click the load balancer name. On the Summary tab of the load balancer details page, find the ID field and copy it.

NOTE:

The system preferentially connects to the load balancer based on the kubernetes.io/elb.id field. If this field is not specified, the spec.loadBalancerIP field is used (optional and available only in 1.23 and earlier versions).

Do not use the spec.loadBalancerIP field to connect to the load balancer. This field will be discarded by Kubernetes. For details, see Deprecation.

v1.9 or later

kubernetes.io/elb.autocreate

Table 24

Mandatory when load balancers are automatically created. This parameter cannot be modified after the Service is created.

CAUTION:

kubernetes.io/elb.id and kubernetes.io/elb.autocreate cannot be specified concurrently.

If both annotations are specified in a Service, CCE will use an existing load balancer instead of automatically creating one. If this Service is deleted, the existing load balancer is considered as an automatically created one. If no other resources are associated with the load balancer, the load balancer will also be deleted.

Example:

  • Automatically created dedicated load balancer with an EIP bound:

    '{"type":"public","bandwidth_name":"cce-bandwidth-1741230802502","bandwidth_chargemode":"bandwidth","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","available_zone":["*****"],"elb_virsubnet_ids":["*****"],"l7_flavor_name":"","l4_flavor_name":"L4_flavor.elb.pro.max","vip_subnet_cidr_id":"*****"}'

  • Automatically created dedicated load balancer with no EIP bound:

    '{"type":"inner","available_zone":["*****"],"elb_virsubnet_ids":["*****"],"l7_flavor_name":"","l4_flavor_name":"L4_flavor.elb.pro.max","vip_subnet_cidr_id":"*****"}'

  • Automatically created shared load balancer with an EIP bound:

    '{"type":"public","bandwidth_name":"cce-bandwidth-1551163379627","bandwidth_chargemode":"bandwidth","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","name":"james"}'

  • Automatically created shared load balancer with no EIP bound:

    {"type":"inner","name":"A-location-d-test"}

v1.9 or later

kubernetes.io/elb.enterpriseID

String

Optional when load balancers are automatically created.

This parameter is available in clusters of v1.15 or later. In clusters earlier than v1.15, load balancers are created in the default project by default.

This parameter indicates the ID of the enterprise project in which the ELB load balancer will be created.

If this parameter is not specified or is set to 0, resources will be bound to the default enterprise project.

How to obtain:

Log in to the EPS console, click the name of the target enterprise project. On the enterprise project details page, find the ID field and copy the ID.

v1.15 or later

kubernetes.io/elb.subnet-id

String

Optional when load balancers are automatically created.

ID of the subnet where the cluster is located. The value can contain 1 to 100 characters.

  • Mandatory when a cluster of v1.11.7-r0 or earlier is to be automatically created.
  • Optional for clusters of a version later than v1.11.7-r0.

Mandatory for clusters of a version earlier than v1.11.7-r0

Discarded in clusters of a version later than v1.11.7-r0

kubernetes.io/elb.lb-algorithm

String

Specifies the load balancing algorithm of the backend server group. The default value is ROUND_ROBIN.

Options:

  • ROUND_ROBIN: weighted round robin algorithm
  • LEAST_CONNECTIONS: weighted least connections algorithm
  • SOURCE_IP: source IP hash algorithm
NOTE:

If this parameter is set to SOURCE_IP, the weight setting (weight field) of backend servers bound to the backend server group is invalid, and sticky session cannot be enabled.

v1.9 or later

The following shows how to use the preceding annotations:

  • Associate an existing load balancer. For details, see Using kubectl to Create a Service (Using an Existing Load Balancer). 
    apiVersion: v1 
kind: Service 
metadata: 
  name: nginx
  annotations:
    kubernetes.io/elb.id: <your_elb_id>                         # Load balancer ID. Replace it with the actual value.
    kubernetes.io/elb.class: performance                        # Load balancer type
    kubernetes.io/elb.lb-algorithm: ROUND_ROBIN                   # Load balancer algorithm
spec:
  selector: 
     app: nginx
  ports: 
  - name: service0 
    port: 80
    protocol: TCP 
    targetPort: 80
  type: LoadBalancer
  • Automatically create a load balancer. For details, see Using kubectl to Create a Service (Automatically Creating a Load Balancer).
    Shared load balancer: 
    apiVersion: v1 
kind: Service 
metadata: 
  annotations:   
    kubernetes.io/elb.class: union
    kubernetes.io/elb.autocreate: '{
      "type": "public",
      "bandwidth_name": "cce-bandwidth-1551163379627",
      "bandwidth_chargemode": "bandwidth",
      "bandwidth_size": 5,
      "bandwidth_sharetype": "PER",
      "eip_type": "5_bgp"
    }'
    kubernetes.io/elb.enterpriseID: '0'               # ID of the enterprise project to which the load balancer belongs
    kubernetes.io/elb.lb-algorithm: ROUND_ROBIN     # Load balancer algorithm
  labels: 
    app: nginx 
  name: nginx 
spec: 
  ports: 
  - name: service0 
    port: 80
    protocol: TCP 
    targetPort: 80
  selector: 
    app: nginx 
  type: LoadBalancer
    Dedicated load balancer: 
    apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
  namespace: default
  annotations:
    kubernetes.io/elb.class: performance
    kubernetes.io/elb.autocreate: '{
      "type": "public",
      "bandwidth_name": "cce-bandwidth-1626694478577",
      "bandwidth_chargemode": "bandwidth",
      "bandwidth_size": 5,
      "bandwidth_sharetype": "PER",
      "eip_type": "5_bgp",
      "available_zone": [
         ""
      ],
      "l4_flavor_name": "L4_flavor.elb.s1.small"
    }'
    kubernetes.io/elb.enterpriseID: '0'               # ID of the enterprise project to which the load balancer belongs
    kubernetes.io/elb.lb-algorithm: ROUND_ROBIN     # Load balancer algorithm
spec:
  selector:
    app: nginx
  ports:
  - name: cce-service-0
    targetPort: 80
    nodePort: 0
    port: 80
    protocol: TCP
  type: LoadBalancer

Sticky Session

Table 2 Annotations for sticky session

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.session-affinity-mode

String

Source IP address-based sticky session means that access requests from the same IP address are forwarded to the same backend server.

  • To disable sticky sessions, leave this parameter unconfigured.
  • To enable sticky session, add this parameter and set it to SOURCE_IP, indicating that the sticky session is based on the source IP address.
NOTE:

When kubernetes.io/elb.lb-algorithm is set to SOURCE_IP (source IP hash), sticky session cannot be enabled.

v1.9 or later

kubernetes.io/elb.session-affinity-option

Table 27

Sticky session timeout.

v1.9 or later
The following shows how to use the preceding annotations: 
apiVersion: v1 
kind: Service 
metadata: 
  name: nginx
  annotations:
    kubernetes.io/elb.id: <your_elb_id>                         # Load balancer ID. Replace it with the actual value.
    kubernetes.io/elb.class: performance                   # Load balancer type
    kubernetes.io/elb.session-affinity-mode: SOURCE_IP          # The sticky session type is source IP address.
    kubernetes.io/elb.session-affinity-option: '{"persistence_timeout": "30"}'     # Stickiness duration, which is measured in minutes
spec:
  selector: 
     app: nginx
  ports: 
  - name: service0 
    port: 80
    protocol: TCP 
    targetPort: 80
  type: LoadBalancer

Health Check

Table 3 Annotations for health check

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.health-check-flag

String

Whether to enable the ELB health check. If this annotation is not added, the health check is enabled by default.

  • Enabling health check: Leave this parameter blank or set it to on.
  • Disabling health check: Set this parameter to off.

If this parameter is set to on, you must specify kubernetes.io/elb.health-check-option or kubernetes.io/elb.health-check-options.

v1.9 or later

kubernetes.io/elb.health-check-option

Table 25

ELB health check configuration option.

v1.9 or later

kubernetes.io/elb.health-check-options

Table 26

ELB health check configuration options. Each Service port can be configured separately, and you can configure only some ports.

NOTE:

Either kubernetes.io/elb.health-check-option or kubernetes.io/elb.health-check-options can be configured.

v1.19.16-r5 or later

v1.21.8-r0 or later

v1.23.6-r0 or later

v1.25.2-r0 or later
  • The following shows how to use kubernetes.io/elb.health-check-option: 
    apiVersion: v1 
kind: Service 
metadata: 
  name: nginx
  annotations:
    kubernetes.io/elb.id: <your_elb_id>                         # Load balancer ID. Replace it with the actual value.
    kubernetes.io/elb.class: performance                   # Load balancer type
    kubernetes.io/elb.health-check-flag: 'on'                   # Enable ELB health check.
    kubernetes.io/elb.health-check-option: '{
      "protocol":"TCP",
      "delay":"5",
      "timeout":"10",
      "max_retries":"3"
    }'
spec:
  selector: 
     app: nginx
  ports: 
  - name: service0 
    port: 80
    protocol: TCP 
    targetPort: 80
  type: LoadBalancer
  • For details about how to use kubernetes.io/elb.health-check-options, see Configuring Health Check on Multiple LoadBalancer Service Ports.

Passthrough Capability

Table 4 Annotations for passthrough capability

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.pass-through

String

Whether the access requests from within the cluster to the Service pass through the ELB load balancer.

v1.19 or later

For details about application scenarios and use cases, see Configuring Passthrough Networking for a LoadBalancer Service.

Resource Tags

Table 5 Annotations

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.tags

String

Add resource tags to a load balancer. This parameter can be configured only when a load balancer is automatically created.

A tag is in the format of "key=value". Use commas (,) to separate multiple tags.

v1.23.11-r0, v1.25.6-r0, v1.27.3-r0, or later

For details about application scenarios and use cases, see Using kubectl to Create a Service (Automatically Creating a Load Balancer).

Changing a Custom EIP

Table 6 Annotations for changing a custom EIP

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.custom-eip-id

String

ID of the custom EIP, which can be seen on the EIP console.

The EIP must be bindable.

v1.23.18-r0, v1.25.13-r0, v1.27.10-r0, v1.28.8-r0, v1.29.4-r0, v1.30.1-r0, or later

For details about application scenarios and use cases, see Changing a Custom EIP for a LoadBalancer Service.

Configuring a Reclaim Policy for an Auto-created Load Balancer

Table 7 Annotation for configuring a reclaim policy for an auto-created load balancer

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.instance-reclaim-policy

String

Optional when load balancers are automatically created.

Configure a reclaim policy for an auto-created load balancer. The options are as follows:

  • (Default) Absence of the parameter or presence as blank: The auto-created load balancer is deleted along with the Service/ingress. However, if the load balancer is associated with multiple Services/ingresses and the Service/ingress that is created along with the load balancer is deleted first, the load balancer may be retained even after all its associated Services/ingresses are deleted.
  • retain: The auto-created load balancer is retained when the Service/ingress is deleted.
  • alwaysDelete: The auto-created load balancer is deleted along with the Service/ingress. If the load balancer is associated with multiple Services/ingresses, it will be deleted only after all its associated Services/ingresses are deleted.
NOTE:
  • If an auto-created load balancer is associated with other resources or its billing mode is changed to yearly/monthly, it will be retained even if the reclaim policy is not set to retain.
  • If an auto-created load balancer is associated with multiple Services/ingresses, this parameter applies only to the Service/ingress that is created along with the load balancer.

v1.28.15-r60, v1.29.15-r20, v1.30.14-r20, v1.31.10-r20, v1.32.6-r20, v1.33.5-r10, or later

The following shows how to use the preceding annotation:

apiVersion: v1
kind: Service
metadata:
  name: service-0
  labels:
    app: test
    version: v1
  namespace: default
  annotations:
    kubernetes.io/elb.class: performance
    kubernetes.io/elb.autocreate: '{"type":"inner","available_zone":["******"],"elb_virsubnet_ids":["******"],"ipv6_vip_virsubnet_id":"******","l7_flavor_name":"","l4_flavor_name":"L4_flavor.elb.pro.max","vip_subnet_cidr_id":"******"}'
    kubernetes.io/elb.enterpriseID: '0'
    kubernetes.io/elb.lb-algorithm: ROUND_ROBIN
    kubernetes.io/elb.health-check-flag: 'on'
    kubernetes.io/elb.health-check-option: '{"protocol":"TCP","delay":"5","timeout":"10","max_retries":"3"}'
    kubernetes.io/elb.instance-reclaim-policy: retain   # Configure a reclaim policy for an auto-created load balancer.
spec:
  selector:
    app: nginx
    version: v1
  externalTrafficPolicy: Cluster
  ports:
    - name: cce-service-0
      targetPort: 80
      nodePort: 0
      port: 80
      protocol: TCP
  type: LoadBalancer
  loadBalancerIP: ''

HTTP/HTTPS/QUIC

Table 8 Annotations for using HTTP or HTTPS

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.protocol-port

String

If a Service is HTTP/HTTPS/QUIC-compliant, configure the protocol and port in the format of protocol:port.

where,

  • protocol: specifies the protocol used by the listener port. The value can be http, quic, or https.
  • ports: Service ports specified by spec.ports[].port.

v1.19.16 or later

v1.28.15-r80, v1.29.15-r40, v1.30.14-r40, v1.31.14-r0, v1.32.9-r0, v1.33.7-r0, v1.34.2-r0, or later (if QUIC is used)

kubernetes.io/elb.cert-id

String

ID of an ELB certificate, which is used as the HTTPS server certificate.

How to obtain: Log in to the ELB console and choose Certificates. In the certificate list, copy the ID under the target certificate name.

v1.19.16 or later

For details about application scenarios and use cases, see Configuring HTTP/HTTPS for a LoadBalancer Service and Configuring a QUIC Listener for a LoadBalancer Service.

SNI

Table 9 Annotations for using SNIs

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.tls-certificate-ids

String

In ELB, the IDs of SNI certificates that must contain a domain name are separated by commas (,).

How to obtain: Log in to the ELB console and choose Certificates. In the certificate list, copy the ID under the target certificate name.

v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later

HTTPS must be enabled. For details, see Configuring SNI for a LoadBalancer Service.

HTTP/2

Table 10 Annotations for using HTTP/2

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.http2-enable

String

Whether HTTP/2 is enabled. Request forwarding using HTTP/2 improves access performance between your application and the load balancer. However, the load balancer still uses HTTP/1.x to forward requests to the backend server.

Options:

  • true: enabled
  • false: disabled (default value)

Note: HTTP/2 can be enabled or disabled only when the listener uses HTTPS. This parameter is invalid and defaults to false when the listener protocol is HTTP.

v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later

For details about application scenarios and use cases, see Configuring HTTP/2 for a LoadBalancer Service.

Configuring Mutual Authentication

Table 11 Annotation for configuring mutual authentication

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.client-ca-cert-id

String

Required only for mutual authentication. The ELB certificate ID serves as the CA certificate.

How to obtain: Log in to the ELB console and choose Certificates. In the certificate list, copy the ID under the target certificate name.

Dedicated load balancers are available in clusters v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later. Shared load balancers are available in clusters v1.28.15-r60, v1.29.15-r20, v1.30.14-r20, v1.31.10-r20, v1.32.6-r20, v1.33.5-r10, or later.

The following shows how to use the preceding annotation:

apiVersion: v1
kind: Service
metadata:
  name: test
  labels:
    app: test
  namespace: default
  annotations:
    kubernetes.io/elb.class: performance
    kubernetes.io/elb.id: *****
    kubernetes.io/elb.protocol-port: https:443
    kubernetes.io/elb.cert-id: *****
    kubernetes.io/elb.client-ca-cert-id: *****   # Configure mutual authentication.
    kubernetes.io/elb.security-pool-protocol: 'off'
    kubernetes.io/elb.tls-ciphers-policy: tls-1-2
    kubernetes.io/elb.lb-algorithm: ROUND_ROBIN
    kubernetes.io/elb.health-check-flag: 'on'
    kubernetes.io/elb.health-check-option: '{"protocol":"TCP","delay":"5","timeout":"10","max_retries":"3"}'
spec:
  selector:
    app: test
  externalTrafficPolicy: Cluster
  ports:
    - name: cce-service-0
      targetPort: 80
      nodePort: 0
      port: 443
      protocol: TCP
  type: LoadBalancer
  loadBalancerIP: **.**.**.**

Configuring the Backend Routing Policy of a Listener

For details about application scenarios and use cases, see Configuring a UDP listener for the QUIC Protocol for a LoadBalancer Service.

Table 12 Annotation for configuring the backend routing policy of a listener

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.pools-options

String

Backend server group configuration, including the backend protocol, load balancing algorithm, and sticky session.

The annotation value is a JSON array. For details, see Table 13.

CAUTION:

After the kubernetes.io/elb.pools-options annotation is configured, the legacy annotation of load balancing algorithm (kubernetes.io/elb.lb-algorithm), sticky session (kubernetes.io/elb.session-affinity-option or kubernetes.io/elb.session-affinity-options), or backend protocol (kubernetes.io/elb.security-pool-protocol) cannot be configured.

v1.28.15-r80, v1.29.15-r40, v1.30.14-r40, v1.31.14-r0, v1.32.9-r0, v1.33.7-r0, v1.34.2-r0, or later

The following is an example of using the preceding annotation:

apiVersion: v1
kind: Service
metadata:
  name: test
  labels:
    app: nginx
  namespace: default
  annotations:
    kubernetes.io/elb.class: performance
    kubernetes.io/elb.id: 95df847a-dec7-4c7f-8e8b-5422344f6153
    kubernetes.io/elb.protocol-port: https:443  
    kubernetes.io/elb.cert-id: d30be9bc114a40099bc2c8e4d1855eb3
    kubernetes.io/elb.pools-options: '[{"protocol":"HTTP","lb_algorithm":"ROUND_ROBIN","target_service_port":"TCP:443"}]'
    kubernetes.io/elb.health-check-flag: 'on'
    kubernetes.io/elb.health-check-option: '{"protocol":"TCP","delay":"5","timeout":"10","max_retries":"3"}'
spec:
  selector:
    app: nginx
  externalTrafficPolicy: Cluster
  ports:
    - name: cce-service-0
      targetPort: 80
      nodePort: 0
      port: 443
      protocol: TCP
  type: LoadBalancer
  loadBalancerIP: **.**.**.**
Table 13 Parameters for configuring a backend server group

Parameter

Mandatory

Type

Description

protocol

Yes

String

Definition: Protocol of a backend server. If the frontend protocol and port remain unchanged, the backend protocol cannot be modified.

Constraints: The custom configuration must match the protocol of the frontend listener. For details about the matching rules, see Mapping Between Frontend and Backend Protocols for Load Balancing and Configuration Examples. No constraints apply to global configurations.

Range: TCP, TLS, UDP, HTTP, HTTPS, or QUIC

Default value: N/A

target_service_port

No

String

Definition: The backend server group to which the configuration applies. The value is composed of the protocol and port defined in spec.ports. For example, TCP:80 applies the configuration to the backend server group using TCP on port 80. If omitted or set to *, the configuration is treated as global and applies to all backend server groups.

Constraints: For global configuration, the array length must be 1. A Service may contain listeners with different frontend protocols, but backend protocol options are limited. If this parameter is set globally and the backend protocol does not match the frontend protocol, the default value will be used. For details about the matching rules, see Mapping Between Frontend and Backend Protocols for Load Balancing and Configuration Examples. Other parameters depend on the backend protocol, so defaults may be applied when mismatches occur.

Range: <protocol>:<port>

Default value: None (Global configuration is used.)

lb_algorithm

Yes

String

Definition: The load balancing algorithm used by a backend server group

Constraints:

  • If set to SOURCE_IP or QUIC_CID, the weights of backend servers in the backend server group are invalid.
  • QUIC_CID is supported only when the backend protocol of the backend server group is QUIC.

Range:

  • ROUND_ROBIN: weighted round robin
  • LEAST_CONNECTIONS: weighted least connections
  • SOURCE_IP: source IP hash
  • QUIC_CID: QUIC connection ID hash

Default value: If the backend protocol is QUIC, the value can only be QUIC_CID. The default value is ROUND_ROBIN for other protocols.

session_persistence

No

SessionPersistence

Definition: The sticky session settings

Constraints: None

Default value: Disabled

quic_cid_hash_strategy

No

QuicCidHashStrategy

Definition: The multi-path forwarding behavior based on destination QUIC connection IDs

Constraints: This parameter can be configured only when lb_algorithm is set to QUIC_CID.

Default value: N/A

Parameters in the session_persistence and quic_cid_hash_strategy fields

  • SessionPersistence

    Parameter

    Mandatory

    Type

    Description

    type

    Yes

    String

    Definition: The sticky session type

    Constraints:

    • For a backend server group using TCP, UDP, or QUIC, the value must be SOURCE_IP.
    • For a backend server group using HTTP or HTTPS, the value must be HTTP_COOKIE.

    Range:

    • SOURCE_IP: Source IP address-based sticky sessions forward requests from the same client IP address to the same backend server.
    • HTTP_COOKIE: Cookie-based sticky sessions insert a dedicated cookie into the HTTP response. The load balancer reads this cookie in subsequent requests and routes them to the corresponding backend server.

    Default value: N/A

    persistence_timeout

    Yes

    Integer

    Definition: The sticky session duration

    Constraints: N/A

    Range:

    • For a backend server group using TCP, UDP, or QUIC: 1 to 60 (minutes)
    • For a backend server group using HTTP or HTTPS: 1 to 1,440 (minutes)

    Default value: N/A
  • QuicCidHashStrategy

    Parameter

    Mandatory

    Type

    Description

    len

    Yes

    Integer

    Definition: The length of a CID segment used for hash-based load balancing

    Constraints: This parameter is valid only when the load balancing algorithm is QUIC_CID.

    Range: 1 to 20 (The sum of len and offset must be ≤ 20.)

    Default value: 3

    offset

    Yes

    Integer

    Definition: The offset within a CID from which the hash calculation begins

    Constraints: This parameter is valid only when the load balancing algorithm is QUIC_CID.

    Range: 0 to 19 (The sum of len and offset must be ≤ 20.)

    Default value: 1

Dynamically Adjusting the Weight of the Backend Server

Table 14 Annotation for dynamically adjusting the weight of the backend server

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.adaptive-weight

String

Dynamically adjust the weight of the load balancer backend server based on the number of pods on the server. In this way, the requests received by each pod are more balanced.

  • true: enabled
  • false: disabled

v1.21 or later

This parameter is invalid in passthrough networking, where dedicated load balancers are used in a CCE Turbo cluster.

The following shows how to use the preceding annotation: 
apiVersion: v1 
kind: Service 
metadata: 
  name: nginx
  annotations:
    kubernetes.io/elb.id: <your_elb_id>                         # Load balancer ID. Replace it with the actual value.
    kubernetes.io/elb.class: performance                   # Load balancer type
    kubernetes.io/elb.adaptive-weight: 'true'                   # Enable dynamic adjustment of the weight of the backend ECS.
spec:
  selector: 
     app: nginx
  ports: 
  - name: service0 
    port: 80
    protocol: TCP 
    targetPort: 80
  type: LoadBalancer

Blocklist/Trustlist

Table 15 Annotations for ELB access control

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.acl-id

String
  • If this parameter is not specified, CCE does not modify access control on the ELB.
  • If this parameter is left empty, all IP addresses are allowed to access the load balancer.
  • If this parameter is set to the IP address group ID of the load balancer, access control is enabled and you need to configure an IP address blocklist or trustlist for the load balancer. Additionally, you need to configure both kubernetes.io/elb.acl-status and kubernetes.io/elb.acl-type.
    NOTE:

    For clusters of v1.25.16-r10, v1.27.16-r10, v1.28.15-r0, v1.29.10-r0, v1.30.6-r0, v1.31.1-r0, or later, when using a dedicated load balancer, you can select a maximum of five IP address groups at a time, separated by commas (,).

    How to obtain:

    Log in to the ELB console, choose Elastic Load Balance > IP Address Groups, and copy the ID of the target IP address group. For details, see Creating an IP Address Group.

v1.23.12-r0, v1.25.7-r0, v1.27.4-r0, v1.28.2-r0, or later

kubernetes.io/elb.acl-status

String

This parameter is mandatory when you configure an IP address blocklist or trustlist for a load balancer. Options:

  • on or true: Access control is enabled.
  • off or false: Access control is disabled.

v1.23.12-r0, v1.25.7-r0, v1.27.4-r0, v1.28.2-r0, or later

kubernetes.io/elb.acl-type

String

This parameter is mandatory when you configure an IP address blocklist or trustlist for a load balancer. Options:

  • black: indicates a blocklist. The selected IP address group cannot access the load balancer.
  • white: indicates a trustlist. Only the selected IP address group can access the load balancer.

v1.23.12-r0, v1.25.7-r0, v1.27.4-r0, v1.28.2-r0, or later
The following shows how to use the preceding annotations: 
apiVersion: v1 
kind: Service 
metadata: 
  name: nginx
  annotations:
    kubernetes.io/elb.id: <your_elb_id>                    # Load balancer ID. Replace it with the actual value.
    kubernetes.io/elb.class: performance                   # Load balancer type
    kubernetes.io/elb.acl-id: <your_acl_id>               # ID of an IP address group for accessing a load balancer
    kubernetes.io/elb.acl-status: 'on'                    # Enable access control.
    kubernetes.io/elb.acl-type: 'white'                   # Trustlist for access control
spec:
  selector: 
     app: nginx
  ports: 
  - name: service0 
    port: 80
    protocol: TCP 
    targetPort: 80
  type: LoadBalancer

For details about application scenarios and use cases, see Configuring a Blocklist/Trustlist Access Policy for a LoadBalancer Service.

Host Network

Table 16 Annotation for host network

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/hws-hostNetwork

String

This annotation can be used by the Service only if the pod uses the host network. After this annotation is used, ELB forwards requests to the host network.

Options:

  • true: enabled
  • false (default): disabled

v1.9 or later
The following shows how to use the preceding annotation: 
apiVersion: v1 
kind: Service 
metadata: 
  name: nginx
  annotations:
    kubernetes.io/elb.id: <your_elb_id>                         # Load balancer ID. Replace it with the actual value.
    kubernetes.io/elb.class: performance                   # Load balancer type
    kubernetes.io/hws-hostNetwork: 'true'                     # The load balancer forwards the request to the host network.
spec:
  selector: 
     app: nginx
  ports: 
  - name: service0 
    port: 80
    protocol: TCP 
    targetPort: 80
  type: LoadBalancer

Timeout

Table 17 Annotation for configuring timeout

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.keepalive_timeout

String

Timeout for client connections. If there are no requests reaching the load balancer during this period, the load balancer will disconnect the connection from the client and establish a new connection when there is a new request.

Value:

  • For TCP listeners, the value ranges from 10 to 4000 (in seconds). The default value is 300.
  • For HTTP, HTTPS, and TERMINATED_HTTPS listeners, the value ranges from 0 to 4000 (in seconds). The default value is 60.
  • For UDP listeners (supported only by dedicated load balancers), the value ranges from 10 to 4000 (in seconds). The default value is 300.

Dedicated load balancers: v1.19.16-r30, v1.21.10-r10, v1.23.8-r10, v1.25.3-r10, or later

Shared load balancers: v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later

kubernetes.io/elb.client_timeout

String

Timeout for waiting for a request from a client. There are two cases:

  • If the client fails to send a request header to the load balancer during this period, the request will be interrupted.
  • If the interval between two consecutive request bodies reaching the load balancer exceeds this period, the connection will be disconnected.

The value ranges from 1 to 300 (in seconds). The default value is 60.

v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later

kubernetes.io/elb.member_timeout

String

Timeout for waiting for a response from a backend server. After a request is forwarded to the backend server, if the backend server does not respond within the value specified by member_timeout, the load balancer will stop waiting and return HTTP 504 Gateway Timeout.

The value ranges from 1 to 300 (in seconds). The default value is 60.

v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later

For details about application scenarios and use cases, see Configuring Timeout for a LoadBalancer Service.

Configuring an HTTP/HTTPS Header

Table 18 Annotations for configuring an HTTP/HTTPS header

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.x-forwarded-port

String

A load balancer can obtain the port number of a listener using X-Forwarded-Port and transmit the port number to the packets of the backend server.

  • true: Enable the function of obtaining a listener port number.
  • false: Disable the function of obtaining a listener port number.

v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later

kubernetes.io/elb.x-forwarded-for-port

String

A load balancer can obtain a client port number for requests using X-Forwarded-For-Port and transmit the port number to the packets of the backend server.

  • true: Enable the function of obtaining a client port number for requests.
  • false: Disable the function of obtaining a client port number for requests.

kubernetes.io/elb.x-forwarded-host

String
  • true: Enable the function of rewriting X-Forwarded-Host. Then, the X-Forwarded-Host header will be rewritten using the Host header of the client request and transmitted to backend servers.
  • false: Disable the function of rewriting X-Forwarded-Host. Then, the X-Forwarded-Host header of the client will be transmitted to backend servers.

kubernetes.io/elb.x-real-ip

String
  • true: Enable the function of rewriting X-Real-IP. The source IP address of the client will be written into the X-Real-IP header and transmitted to the backend servers.
  • false: Disable the function of rewriting X-Real-IP. The X-Real-IP header of the client will be transmitted to the backend servers.

v1.25.16-r30, v1.27.16-r30, v1.28.15-r20, v1.29.13-r0, v1.30.10-r0, v1.31.6-r0, v1.32.1-r0, or later

For details about application scenarios and use cases, see Configuring an HTTP/HTTPS Header for a LoadBalancer Service.

Enabling GZIP

Table 19 Annotations for enabling GZIP

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.gzip-enabled

String

LoadBalancer Services support data compression, which reduces the size of files to be transferred, improves file transfer efficiency, and reduces the bandwidth needed for the transmission.

If this function is enabled, specific files will be compressed. If it is not enabled, files will not be compressed. By default, data compression is disabled.

The files in the following format can be compressed:

  • Brotli can compress all file formats.
  • GZIP can compress the files of the following types: text/xml, text/plain, text/css, application/javascript, application/x-javascript, application/rss+xml, application/atom+xml, application/xml, and application/json

This function is available only for HTTP/HTTPS listeners of dedicated load balancers. If the advanced configuration for enabling data compression or the target annotation is deleted, the ELB configuration will not be modified.

v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later

For details about application scenarios and use cases, see Configuring GZIP Data Compression for a LoadBalancer Service.

Configuring Graceful Exit for the Load Balancer Backend

You can configure the graceful exit time for the backend servers of load balancers associated with TCP- or UDP-compliant Services. Graceful exit is set to 300s by default.

ELB does not offer this feature to everyone. This feature is only supported by dedicated load balancers. To use this feature, submit a service ticket to ELB.

Table 20 Annotations for the graceful exit of the load balancer backend

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.connection-drain-enable

String

Whether to enable graceful exit for the load balancer backend.

  • true: This function is enabled.
  • false: This function is disabled.

v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later

kubernetes.io/elb.connection-drain-timeout

String

Graceful exit time of the load balancer backend. The value ranges from 10 to 4000 (in seconds). The default value is 300.

Obtaining Client IP Addresses

Table 21 Annotations for obtaining client IP addresses

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.transparent-client-ip

String

This parameter is available only when a shared load balancer is used to create a LoadBalancer Service that is TCP/UDP-compliant.

  • true: Enabling the function of obtaining the client source IP address.
  • false: Disabling the function of obtaining the client source IP address.

v1.23.17-r0, v1.25.12-r0, v1.27.9-r0, v1.28.7-r0, v1.29.3-r0, or later

For details about application scenarios and use cases, see Enabling a LoadBalancer Service to Obtain the Client IP Address.

Configuring Port Ranges for a Listener

Table 22 Annotations for configuring port ranges for a listener

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.port-ranges

String

If a dedicated load balancer is used and the TCP, UDP, or TLS protocol is selected, you can create a listener that listens to ports within a certain range from 1 to 65535. You can add a maximum of 10 port ranges that do not overlap for each listener.

The parameter value is in the following format, where ports_name and port must be unique:

 '{"<ports_name_1>":["<port_1>,<port_2>","<port_3>,<port_4>"], "<ports_name_2>":["<port_5>,<port_6>","<port_7>,<port_8>"]}'

For example, the port names are cce-service-0 and cce-service-1, and the listener listens to ports 100–200 and 300–400, and 500–600 and 700–800, respectively.

'{"cce-service-0":["100,200", "300,400"], "cce-service-1":["500,600", "700,800"]}'
NOTE:

This function requires ELB. Before using this function, check whether ELB supports full-port listening and forwarding for layer-4 protocols in the current region.

v1.23.18-r0, v1.25.13-r0, v1.27.10-r0, v1.28.8-r0, v1.29.4-r0, v1.30.1-r0, or later

For details about application scenarios and use cases, see Configuring a Range of Listening Ports for LoadBalancer Services.

Configuring IP Addresses as Backend Servers

Table 23 Annotation for configuring IP addresses as backend servers

Parameter

Type

Description

Supported Cluster Version

kubernetes.io/elb.ip-target-enabled

String
Enable IP addresses as backend servers for a load balancer. After this function is enabled, the backend IP addresses added to the associated Service or ingress are all IP addresses in VPC subnets. For a CCE Turbo cluster, the backend is <pod-IP-address>:<target-port>. For a CCE standard cluster, the backend is <node-IP-address>:<NodePort>.
  • true: Enable IP addresses as backend servers.
  • Other values are not supported.
CAUTION:
  • Only dedicated load balancers support this function.
  • This function can only be enabled during Service or ingress creation. The annotation cannot be added to or removed from an existing Service or ingress. To use this function, you must create a Service or an ingress.
  • If the load balancer associated with a Service or ingress does not have IP as a backend enabled, CCE will automatically enable it after the annotation is added. Once enabled, IP as a backend cannot be disabled.
  • Ensure that the backend subnets of the load balancer associated with the Service or ingress have sufficient IP addresses, or this function cannot be used. If the IP addresses are not enough, you can add more backend subnets on the Summary page of the load balancer.
  • For more details, see Using IP as a Backend to Route Traffic Across Backend Servers.

v1.25.16-r30, v1.27.16-r30, v1.28.15-r20, v1.29.13-r0, v1.30.10-r0, v1.31.6-r0, v1.32.1-r0, or later

For details about application scenarios and use cases, see Configuring IP Addresses as Backend Servers for a LoadBalancer Service.

Parameters for Automatically Creating a Load Balancer

Table 24 elb.autocreate data structure

Parameter

Mandatory

Type

Description

name

No

String

Name of the automatically created load balancer.

The value can contain 1 to 64 characters. Only letters, digits, underscores (_), hyphens (-), and periods (.) are allowed.

Default: cce-lb+service.UID

type

No

String

Network type of the load balancer.

  • public: public network load balancer
  • inner: private network load balancer

Default: inner

bandwidth_name

Yes for public network load balancers

String

Bandwidth name. The default value is cce-bandwidth-******.

The value can contain 1 to 64 characters. Only letters, digits, underscores (_), hyphens (-), and periods (.) are allowed.

bandwidth_chargemode

Yes for public network load balancers

String

Bandwidth billing mode.

  • bandwidth: billed by bandwidth
  • traffic: billed by traffic

bandwidth_size

Yes for public network load balancers

Integer

Bandwidth size. The value ranges from 1 Mbit/s to 2000 Mbit/s by default. Configure this parameter based on the bandwidth range allowed in your region.

The minimum increment for bandwidth adjustment varies depending on the bandwidth range.
  • The minimum increment is 1 Mbit/s if the allowed bandwidth does not exceed 300 Mbit/s.
  • The minimum increment is 50 Mbit/s if the allowed bandwidth ranges from 300 Mbit/s to 1000 Mbit/s.
  • The minimum increment is 500 Mbit/s if the allowed bandwidth exceeds 1000 Mbit/s.

bandwidth_sharetype

Yes for public network load balancers

String

Bandwidth sharing mode.

  • PER: dedicated bandwidth

eip_type

Yes for public network load balancers

String

EIP type.

  • 5_telcom: China Telecom
  • 5_union: China Unicom
  • 5_bgp: dynamic BGP
  • 5_sbgp: static BGP

The types vary by region. For details, see the EIP console.

vip_subnet_cidr_id

No

String

The ID of the IPv4 subnet where the load balancer resides. This subnet is used to allocate IP addresses for the load balancer to provide external services. The IPv4 subnet must belong to the cluster's VPC.

If this parameter is not specified, the load balancer and the cluster will be in the same subnet by default.

This field can be specified only for clusters of v1.21 or later.

How to obtain:

Log in to the VPC console. In the navigation pane, choose Subnets. Filter the target subnet by the cluster's VPC name, click the subnet name, and copy the IPv4 Subnet ID on the Summary tab.

ipv6_vip_virsubnet_id

No

String

The ID of the IPv6 subnet where the load balancer is deployed. IPv6 must be enabled for the subnet.

This parameter is available only for dedicated load balancers.

How to obtain:

Log in to the VPC console. In the navigation pane, choose Subnets. Filter the target subnet by the cluster's VPC name, click the subnet name, and copy the Network ID on the Summary tab.

elb_virsubnet_ids

No

Array of strings

The network ID of the subnet where the load balancer is located. This subnet is used to allocate IP addresses for accessing the backend server. If this parameter is not specified, the subnet specified by vip_subnet_cidr_id will be used by default. Load balancers occupy varying numbers of subnet IP addresses based on their specifications. Do not use the subnet CIDR blocks of other resources (such as clusters or nodes) as the load balancer's CIDR block.

This parameter is available only for dedicated load balancers.

Example:

"elb_virsubnet_ids": [
   "14567f27-8ae4-42b8-ae47-9f847a4690dd"
 ]

How to obtain:

Log in to the VPC console. In the navigation pane, choose Subnets. Filter the target subnet by the cluster's VPC name, click the subnet name, and copy the Network ID on the Summary tab.

vip_address

No

String

Private IP address of the load balancer. Only IPv4 addresses are supported.

The IP address must be in the ELB CIDR block. If this parameter is not specified, an IP address will be automatically assigned from the ELB CIDR block.

This parameter is available only in clusters of v1.23.11-r0, v1.25.6-r0, v1.27.3-r0, or later versions.

available_zone

Yes

Array of strings

AZ where the load balancer is located.

You can obtain all supported AZs by getting the AZ list.

This parameter is available only for dedicated load balancers.

l4_flavor_name

No

String

Flavor name of the Layer 4 load balancer. This parameter is mandatory when TCP, TLS, or UDP is used.

You can obtain all supported types by getting the flavor list.

This parameter is available only for dedicated load balancers.

l7_flavor_name

No

String

Flavor name of the Layer 7 load balancer. This parameter is mandatory when HTTP is used.

You can obtain all supported types by getting the flavor list.

This parameter is available only for dedicated load balancers. Its value must match that of l4_flavor_name, both of which must be either elastic flavors or fixed flavors.
Table 25 elb.health-check-option data structure

Parameter

Mandatory

Type

Description

delay

No

String

Health check interval (s)

Value range: 1 to 50. Default value: 5

timeout

No

String

Health check timeout, in seconds.

Value range: 1 to 50. Default value: 10

max_retries

No

String

Maximum number of health check retries.

Value range: 1 to 10. Default value: 3

protocol

No

String

Health check protocol.

Options: TCP, UDP, or HTTP

path

No

String

Health check URL. This parameter needs to be configured when the protocol is HTTP.

Default value: /

Value range: 1-80 characters
Table 26 elb.health-check-options

Parameter

Mandatory

Type

Description

target_service_port

Yes

String

Port for health check specified by spec.ports. The value consists of the protocol and port number, for example, TCP:80.

monitor_port

No

String

Re-specified port for health check. If this parameter is not specified, the service port is used by default.

NOTE:

Ensure that the port is in the listening state on the node where the pod is located. Otherwise, the health check result will be affected.

delay

No

String

Health check interval (s)

Value range: 1 to 50. Default value: 5

timeout

No

String

Health check timeout, in seconds.

Value range: 1 to 50. Default value: 10

max_retries

No

String

Maximum number of health check retries.

Value range: 1 to 10. Default value: 3

protocol

No

String

Health check protocol.

Default value: protocol of the associated Service

Options: TCP, UDP, and HTTP

path

No

String

Health check URL. This parameter needs to be configured when the protocol is HTTP.

Default value: /

Value range: 1-80 characters

expected_codes

No

String

Expected response status code. This parameter is supported only by clusters of v1.19.16-r50, v1.21.11-r10, v1.23.9-r10, v1.25.4-r10, v1.27.1-r10, or later versions.

Value:

  • A specific value, for example, 200
  • A list of values that are separated with commas (,), for example, 200, 202
  • A value range, for example, 200–204

The default value is 200. The value must contain 1 to 64 characters.

This parameter is available only for HTTP.
Table 27 elb.session-affinity-option data structure

Parameter

Mandatory

Type

Description

persistence_timeout

Yes

String

Sticky session timeout, in minutes. This parameter is valid only when elb.session-affinity-mode is set to SOURCE_IP.

Value range: 1 to 60. Default value: 60
Parent Topic: LoadBalancer