Configuring Advanced LoadBalancer Ingress Functions Using Annotations

Load balancer configuration

• Basic Configurations for Interconnecting with ELB
• Adding Resource Tags
• Configuring a Custom EIP

Port or protocol configuration

• Configuring ELB Certificates
• Using HTTP/2
• Interconnecting with HTTPS Backend Services

Advanced features of ELB listeners

• Configuring Timeout for an Ingress
• Configuring a Slow Start

kubernetes.io/elb.class

String

Select a proper load balancer type.

Options:

• performance: dedicated load balancer, which can be used only in clusters of v1.17 and later.

v1.9 or later

kubernetes.io/ingress.class

String

• cce: A proprietary LoadBalancer ingress is used.
• nginx: indicates that Nginx Ingress is used. This option is available only after NGINX Ingress Controller is installed. For details, see Creating an Nginx Ingress Using kubectl.

  Multiple NGINX Ingress Controller add-ons can be installed in a single cluster if the add-on version is 2.5.4 or later. In this case, the value of this parameter must be the controller name customized during controller installation, indicating that the ingress is managed by that specific controller.

This parameter is mandatory when you create an ingress. The default value is cce.

For clusters of v1.23 or later, use the parameter ingressClassName. For details, see Creating a LoadBalancer Ingress Using kubectl.

Only clusters of v1.21 or earlier

kubernetes.io/elb.port

String

This parameter indicates the external port registered with the address of the LoadBalancer Service.

The value ranges from 1 to 65535. The default value is 80 for HTTP and 443 for HTTPS.

v1.9 or later

kubernetes.io/elb.id

String

When associating only existing load balancers, you can use either this parameter or kubernetes.io/elb.ip. If they conflict, kubernetes.io/elb.id will take precedence.

This parameter indicates the ID of a load balancer.

How to obtain:

Log in to the ELB console and click the load balancer name. On the Summary tab of the load balancer details page, find the ID field and copy it.

v1.9 or later

kubernetes.io/elb.ip

String

When associating only existing load balancers, you can use either this parameter or kubernetes.io/elb.id. If they conflict, kubernetes.io/elb.id will take precedence.

This parameter indicates the IP address of a load balancer.

When creating an ingress, specify a public or private IP address for a shared load balancer. For a dedicated load balancer, only a private IP address can be specified.

v1.9 or later

kubernetes.io/elb.autocreate

Table 12 object

Mandatory when load balancers are automatically created.

Example:

• Automatically created dedicated load balancer with an EIP bound:

  '{"type":"public","bandwidth_name":"cce-bandwidth-1741230802502","bandwidth_chargemode":"bandwidth","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","available_zone":["*****"],"elb_virsubnet_ids":["*****"],"l7_flavor_name":"L7_flavor.elb.pro.max","l4_flavor_name":"","vip_subnet_cidr_id":"*****"}'

• Automatically created dedicated load balancer with no EIP bound:

  '{"type":"inner","available_zone":["*****"],"elb_virsubnet_ids":["*****"],"l7_flavor_name":"L7_flavor.elb.pro.max","l4_flavor_name":"","vip_subnet_cidr_id":"*****"}'

• Automatically created shared load balancer with an EIP bound:

  '{"type":"public","bandwidth_name":"cce-bandwidth-1551163379627","bandwidth_chargemode":"bandwidth","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","name":"james"}'

• Automatically created shared load balancer with no EIP bound:

  '{"type":"inner", "name": "A-location-d-test"}'

v1.9 or later

kubernetes.io/elb.enterpriseID

String

Optional when load balancers are automatically created.

This parameter is available in clusters of v1.15 or later. In clusters earlier than v1.15, load balancers are created in the default project by default.

This parameter indicates the ID of the enterprise project in which the ELB load balancer will be created.

If this parameter is not specified or is set to 0, resources will be bound to the default enterprise project.

How to obtain:

Log in to the EPS console, click the name of the target enterprise project. On the enterprise project details page, find the ID field and copy the ID.

v1.15 or later

kubernetes.io/elb.subnet-id

String

Optional when load balancers are automatically created.

ID of the subnet where the cluster is located. The value can contain 1 to 100 characters.

• Mandatory when a cluster of v1.11.7-r0 or earlier is to be automatically created.
• Optional for clusters of a version later than v1.11.7-r0.

Mandatory for clusters of a version earlier than v1.11.7-r0

Discarded in clusters of a version later than v1.11.7-r0

kubernetes.io/elb.tls-certificate-ids

String

ELB certificate IDs, which are separated by comma (,). The list length is greater than or equal to 1. The first ID in the list is the server certificate, and the other IDs are SNI certificates in which a domain name must be contained.

How to obtain: Log in to the ELB console and choose Certificates. In the load balancer list, copy the ID under the target certificate name.

v1.19.16-r2, v1.21.5-r0, v1.23.3-r0, or later

kubernetes.io/elb.tls-ciphers-policy

String

The default value is tls-1-2, which is the default security policy used by the listener and takes effect only when HTTPS is used.

Options:

• tls-1-0
• tls-1-1
• tls-1-2
• tls-1-2-strict
• tls-1-0-with-1-3 (dedicated load balancer)
• tls-1-2-fs (dedicated load balancer)
• tls-1-2-fs-with-1-3 (dedicated load balancer)

For details of cipher suites for each security policy, see Table 3.

Clusters of v1.17.17 or later

kubernetes.io/elb.security_policy_id

String

The ID of the custom security group policy on ELB. Obtain it on the ELB console. This field takes effect only when HTTPS is used and has a higher priority than the default security policy.

For details about how to create and update a custom security policy, see TLS Security Policy.

Clusters of v1.17.17 or later

kubernetes.io/elb.tags

String

Add resource tags to a load balancer. This parameter can be configured only when a load balancer is automatically created.

A tag is in the format of "key=value". Use commas (,) to separate multiple tags.

, or later

kubernetes.io/elb.http2-enable

String

Whether HTTP/2 is enabled. Request forwarding using HTTP/2 improves the access performance between your application and the load balancer. However, the load balancer still uses HTTP/1.x to forward requests to the backend server.

Options:

• true: enabled
• false: disabled (default value)

Note: HTTP/2 can be enabled or disabled only when the listener uses HTTPS. This parameter is invalid and defaults to false when the listener protocol is HTTP.

v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later

kubernetes.io/elb.pool-protocol

String

To interconnect with HTTPS backend services, set this parameter to https.

v1.23.8, v1.25.3, or later

kubernetes.io/elb.keepalive_timeout

String

Timeout for client connections. If there are no requests reaching the load balancer during the timeout duration, the load balancer will disconnect the connection from the client and establish a new connection when there is a new request.

The value ranges from 0 to 4000 (in seconds). The default value is 60.

Dedicated load balancers: v1.19.16-r30, v1.21.10-r10, v1.23.8-r10, v1.25.3-r10, or later

Shared load balancers: v1.23.13-r0, v1.25.8-r0, v1.28.3-r0, or later

kubernetes.io/elb.client_timeout

String

Timeout for waiting for a request from a client. There are two cases:

• If the client fails to send a request header to the load balancer during the timeout duration, the request will be interrupted.
• If the interval between two consecutive request bodies reaching the load balancer is greater than the timeout duration, the connection will be disconnected.

The value ranges from 1 to 300 (in seconds). The default value is 60.

kubernetes.io/elb.member_timeout

String

Timeout for waiting for a response from a backend server. After a request is forwarded to the backend server, if the backend server does not respond within the duration specified by member_timeout, the load balancer will stop waiting and return HTTP 504 Gateway Timeout.

The value ranges from 1 to 300 (in seconds). The default value is 60.

kubernetes.io/elb.slowstart

String

Duration of slow start, in seconds.

The slow start duration ranges from 30 to 1200.

• This configuration applies only to dedicated load balancers.
• This parameter is valid only when the allocation policy of the target Service is weighted round robin (WRR) and sticky session is disabled.

v1.23 or later

kubernetes.io/elb.ingress-order

String

Specifies the sequence of forwarding rules of different ingresses. The value ranges from 1 to 1000. A smaller value indicates a higher priority. The priority of a forwarding rule must be unique under the same load balancer listener.

This parameter is available only for dedicated load balancers.

v1.23.15-r0, v1.25.10-r0, v1.27.7-r0, v1.28.5-r0, v1.29.1-r10, or later

kubernetes.io/elb.rule-priority-enabled

String

This parameter can only be set to true, indicating to sort the forwarding rules of an ingress. The priorities of the forwarding rules are determined based on the sequence in which they are added during ingress creation. A forwarding rule added earlier indicates a higher priority.

If this parameter is not configured, the default sorting of the forwarding rules on the load balancer will be used. After this parameter is enabled, it cannot be disabled.

This parameter is available only for dedicated load balancers.

kubernetes.io/elb.headers.${svc_name}

String

Custom header of the Service associated with an ingress. ${svc_name} is the Service name.

Format: a JSON string, for example, {"key": "test", "values": ["value1", "value2"]}

• A key-value pair represents the key and value of a custom header. You can configure up to nine values. However, if a header with the key host is specified, only eight values can be configured.

  Enter 1 to 40 characters for a key. Only letters, digits, hyphens (-), and underscores (_) are allowed.

  Enter 1 to 128 characters for a value. Asterisks (*) and question marks (?) are allowed, but spaces and double quotation marks are not allowed. An asterisk can match zero or more characters, and a question mark can match one character.

• Either a custom header or grayscale release can be configured.
• Enter 1 to 51 characters for ${svc_name}.

v1.23.16-r0, v1.25.11-r0, v1.27.8-r0, v1.28.6-r0, v1.29.2-r0, or later

kubernetes.io/elb.custom-eip-id

String

ID of the custom EIP, which can be seen on the EIP console.

The EIP must be bindable.

v1.23.18-r0, v1.25.13-r0, v1.27.10-r0, v1.28.8-r0, v1.29.4-r0, v1.30.1-r0, or later

kubernetes.io/elb.conditions.${svc_name}

String

Configure an advanced forwarding rule. ${svc_name} indicates the Service name, which can contain a maximum of 51 characters.

If the annotation value is set to [], the advanced forwarding rule will be deleted.

The annotation value is a JSON array. For details, see Table 4.

v1.23.18-r10, v1.25.16-r0, v1.27.16-r0, v1.28.13-r0, v1.29.8-r0, v1.30.4-r0, or later

name

No

String

Name of the automatically created load balancer.

The value can contain 1 to 64 characters. Only letters, digits, underscores (_), hyphens (-), and periods (.) are allowed.

Default: cce-lb+service.UID

type

No

String

Network type of the load balancer.

• public: public network load balancer
• inner: private network load balancer

Default: inner

bandwidth_name

Yes for public network load balancers

String

Bandwidth name. The default value is cce-bandwidth-******.

The value can contain 1 to 64 characters. Only letters, digits, underscores (_), hyphens (-), and periods (.) are allowed.

bandwidth_chargemode

Yes for public network load balancers

String

Bandwidth mode.

• bandwidth: billed by bandwidth
• traffic: billed by traffic

bandwidth_size

Yes for public network load balancers

Integer

Bandwidth size. The value ranges from 1 Mbit/s to 2000 Mbit/s by default. Configure this parameter based on the bandwidth range allowed in your region.

bandwidth_sharetype

Yes for public network load balancers

String

Bandwidth sharing mode.

• PER: dedicated bandwidth

eip_type

Yes for public network load balancers

String

EIP type.

• 5_bgp: dynamic BGP
• 5_sbgp: static BGP

The types vary by region. For details, see the EIP console.

vip_subnet_cidr_id

No

String

The ID of the IPv4 subnet where the load balancer resides. This subnet is used to allocate IP addresses for the load balancer to provide external services. The IPv4 subnet must belong to the cluster's VPC.

If this parameter is not specified, the load balancer and the cluster will be in the same subnet by default.

This field can be specified only for clusters of v1.21 or later.

How to obtain:

Log in to the VPC console. In the navigation pane, choose Subnets. Filter the target subnet by the cluster's VPC name, click the subnet name, and copy the IPv4 Subnet ID on the Summary tab.

elb_virsubnet_ids

No

Array of strings

The network ID of the subnet where the load balancer is located. This subnet is used to allocate IP addresses for accessing the backend server. If this parameter is not specified, the subnet specified by vip_subnet_cidr_id will be used by default. Load balancers occupy varying numbers of subnet IP addresses based on their specifications. Do not use the subnet CIDR blocks of other resources (such as clusters or nodes) as the load balancer's CIDR block.

This parameter is available only for dedicated load balancers.

Example:

"elb_virsubnet_ids": [
   "14567f27-8ae4-42b8-ae47-9f847a4690dd"
 ]

How to obtain:

Log in to the VPC console. In the navigation pane, choose Subnets. Filter the target subnet by the cluster's VPC name, click the subnet name, and copy the Network ID on the Summary tab.

vip_address

No

String

Private IP address of the load balancer. Only IPv4 addresses are supported.

The IP address must be in the ELB CIDR block. If this parameter is not specified, an IP address will be automatically assigned from the ELB CIDR block.

This parameter is available only in clusters of v1.23.11-r0, v1.25.6-r0, v1.27.3-r0, or later versions.

available_zone

Yes

Array of strings

AZ where the load balancer is located.

This parameter is available only for dedicated load balancers.

l4_flavor_name

No

String

Flavor name of the Layer 4 load balancer. This parameter is mandatory when TCP or UDP is used.

This parameter is available only for dedicated load balancers.

l7_flavor_name

No

String

Flavor name of the Layer 7 load balancer. This parameter is mandatory when HTTP is used.

This parameter is available only for dedicated load balancers. Its value must match that of l4_flavor_name, meaning both must be either elastic specifications or fixed specifications.