Configuring LoadBalancer Services Using Annotations
You can add annotations to a YAML file to use some CCE advanced functions. This section describes the available annotations when a LoadBalancer service is created.
- Interconnection with ELB
- Sticky Session
- Health Check
- HTTP or HTTPS
- SNI
- Dynamic Adjustment of the Weight of the Backend ECS
- Passthrough Capability
- Blocklist/Trustlist
- Host Network
- Timeout
- Resource Tags
- HTTP/2
- Enabling GZIP
- Obtaining Client IP Addresses
- Configuring a Custom EIP
- Configuring Port Ranges for a Listener
Interconnection with ELB
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.class |
String |
Select a proper load balancer type. Options:
|
v1.9 or later |
kubernetes.io/elb.id |
String |
Mandatory when an existing load balancer is to be associated. ID of a load balancer. How to obtain: On the management console, click Service List, and choose Networking > Elastic Load Balance. Click the name of the target load balancer. On the Summary tab page, find and copy the ID.
NOTE:
The system preferentially connects to the load balancer based on the kubernetes.io/elb.id field. If this field is not specified, the spec.loadBalancerIP field is used (optional and available only in 1.23 and earlier versions). Do not use the spec.loadBalancerIP field to connect to the load balancer. This field will be discarded by Kubernetes. For details, see Deprecation. |
v1.9 or later |
kubernetes.io/elb.autocreate |
Mandatory when load balancers are automatically created. Example
|
v1.9 or later |
|
kubernetes.io/elb.enterpriseID |
String |
Optional when load balancers are automatically created. Clusters of v1.15 and later versions support this field. In clusters earlier than v1.15, load balancers are created in the default project by default. This parameter indicates the ID of the enterprise project in which the ELB load balancer will be created. If this parameter is not specified or is set to 0, resources will be bound to the default enterprise project. How to obtain: Log in to the management console and choose Enterprise > Project Management on the top menu bar. In the list displayed, click the name of the target enterprise project, and copy the ID on the enterprise project details page. |
v1.15 or later |
kubernetes.io/elb.subnet-id |
String |
Optional when load balancers are automatically created. ID of the subnet where the cluster is located. The value can contain 1 to 100 characters.
|
Mandatory for clusters earlier than v1.11.7-r0 Discarded in clusters later than v1.11.7-r0 |
kubernetes.io/elb.lb-algorithm |
String |
Specifies the load balancing algorithm of the backend server group. The default value is ROUND_ROBIN. Options:
NOTE:
If this parameter is set to SOURCE_IP, the weight setting (weight field) of backend servers bound to the backend server group is invalid, and sticky session cannot be enabled. |
v1.9 or later |
The following shows how to use the preceding annotations:
- Associate an existing load balancer. For details, see Using kubectl to Create a Service (Using an Existing Load Balancer).
apiVersion: v1 kind: Service metadata: name: nginx annotations: kubernetes.io/elb.id: <your_elb_id> # Load balancer ID. Replace it with the actual value. kubernetes.io/elb.class: performance # Load balancer type kubernetes.io/elb.lb-algorithm: ROUND_ROBIN # Load balancer algorithm spec: selector: app: nginx ports: - name: service0 port: 80 protocol: TCP targetPort: 80 type: LoadBalancer
- Automatically create a load balancer. For details, see Using kubectl to Create a Service (Automatically Creating a Load Balancer).
Shared load balancer:
apiVersion: v1 kind: Service metadata: annotations: kubernetes.io/elb.class: union kubernetes.io/elb.autocreate: '{ "type": "public", "bandwidth_name": "cce-bandwidth-1551163379627", "bandwidth_chargemode": "bandwidth", "bandwidth_size": 5, "bandwidth_sharetype": "PER", "eip_type": "5_bgp" }' kubernetes.io/elb.enterpriseID: '0' # ID of the enterprise project to which the load balancer belongs kubernetes.io/elb.lb-algorithm: ROUND_ROBIN # Load balancer algorithm labels: app: nginx name: nginx spec: ports: - name: service0 port: 80 protocol: TCP targetPort: 80 selector: app: nginx type: LoadBalancer
Dedicated load balancer:apiVersion: v1 kind: Service metadata: name: nginx labels: app: nginx namespace: default annotations: kubernetes.io/elb.class: performance kubernetes.io/elb.autocreate: '{ "type": "public", "bandwidth_name": "cce-bandwidth-1626694478577", "bandwidth_chargemode": "bandwidth", "bandwidth_size": 5, "bandwidth_sharetype": "PER", "eip_type": "5_bgp", "available_zone": [ "" ], "l4_flavor_name": "L4_flavor.elb.s1.small" }' kubernetes.io/elb.enterpriseID: '0' # ID of the enterprise project to which the load balancer belongs kubernetes.io/elb.lb-algorithm: ROUND_ROBIN # Load balancer algorithm spec: selector: app: nginx ports: - name: cce-service-0 targetPort: 80 nodePort: 0 port: 80 protocol: TCP type: LoadBalancer
Sticky Session
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.session-affinity-mode |
String |
Source IP address-based sticky session means that access requests from the same IP address are forwarded to the same backend server.
NOTE:
When kubernetes.io/elb.lb-algorithm is set to SOURCE_IP (source IP hash), sticky session cannot be enabled. |
v1.9 or later |
kubernetes.io/elb.session-affinity-option |
Sticky session timeout. |
v1.9 or later |
apiVersion: v1 kind: Service metadata: name: nginx annotations: kubernetes.io/elb.id: <your_elb_id> # Load balancer ID. Replace it with the actual value. kubernetes.io/elb.class: performance # Load balancer type kubernetes.io/elb.session-affinity-mode: SOURCE_IP # The sticky session type is source IP address. kubernetes.io/elb.session-affinity-option: '{"persistence_timeout": "30"}' # Stickiness duration (min) spec: selector: app: nginx ports: - name: service0 port: 80 protocol: TCP targetPort: 80 type: LoadBalancer
Health Check
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.health-check-flag |
String |
Whether to enable the ELB health check.
If this parameter is enabled, the kubernetes.io/elb.health-check-option field must also be specified at the same time. |
v1.9 or later |
kubernetes.io/elb.health-check-option |
ELB health check configuration items. |
v1.9 or later |
|
kubernetes.io/elb.health-check-options |
ELB health check configuration items. Each Service port can be configured separately, and you can configure only some ports.
NOTE:
Either kubernetes.io/elb.health-check-option or kubernetes.io/elb.health-check-options can be configured. |
v1.19.16-r5 or later v1.21.8-r0 or later v1.23.6-r0 or later v1.25.2-r0 or later |
- The following shows how to use kubernetes.io/elb.health-check-option:
apiVersion: v1 kind: Service metadata: name: nginx annotations: kubernetes.io/elb.id: <your_elb_id> # Load balancer ID. Replace it with the actual value. kubernetes.io/elb.class: performance # Load balancer type kubernetes.io/elb.health-check-flag: 'on' # Enable ELB health check. kubernetes.io/elb.health-check-option: '{ "protocol":"TCP", "delay":"5", "timeout":"10", "max_retries":"3" }' spec: selector: app: nginx ports: - name: service0 port: 80 protocol: TCP targetPort: 80 type: LoadBalancer
- For details about how to use kubernetes.io/elb.health-check-options, see Configuring Health Check on Multiple Ports of a LoadBalancer Service.
HTTP or HTTPS
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.protocol-port |
String |
If a Service is HTTP/HTTPS-compliant, configure the protocol and port number in the format of "protocol:port". where,
|
v1.19.16 or later |
kubernetes.io/elb.cert-id |
String |
ID of an ELB certificate, which is used as the HTTPS server certificate. To obtain the certificate, log in to the CCE console, choose Service List > Networking > Elastic Load Balance, and click Certificates in the navigation pane. In the load balancer list, copy the ID under the target certificate name. |
v1.19.16 or later |
For details, see Configuring HTTP/HTTPS for a LoadBalancer Service.
SNI
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.tls-certificate-ids |
String |
In ELB, the IDs of SNI certificates that must contain a domain name are separated by commas (,). To obtain the certificate, log in to the CCE console, choose Service List > Networking > Elastic Load Balance, and click Certificates in the navigation pane. In the load balancer list, copy the ID under the target certificate name. |
v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later |
HTTPS must be enabled. For details, see Configuring SNI for a LoadBalancer Service.
Dynamic Adjustment of the Weight of the Backend ECS
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.adaptive-weight |
String |
Dynamically adjust the weight of the load balancer backend server based on the number pods on the server. In this way, the requests received by each pod are more balanced.
|
v1.21 or later |
This parameter is invalid in passthrough networking, where dedicated load balancers are used in a CCE Turbo cluster.
apiVersion: v1 kind: Service metadata: name: nginx annotations: kubernetes.io/elb.id: <your_elb_id> # Load balancer ID. Replace it with the actual value. kubernetes.io/elb.class: performance # Load balancer type kubernetes.io/elb.adaptive-weight: 'true' # Enable dynamic adjustment of the weight of the backend ECS. spec: selector: app: nginx ports: - name: service0 port: 80 protocol: TCP targetPort: 80 type: LoadBalancer
Passthrough Capability
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.pass-through |
String |
Whether the access requests from within the cluster to the Service pass through the ELB load balancer. |
v1.19 or later |
For details, see Configuring Passthrough Networking for a LoadBalancer Service.
Blocklist/Trustlist
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.acl-id |
String |
|
v1.23.12-r0, v1.25.7-r0, v1.27.4-r0, v1.28.2-r0, or later |
kubernetes.io/elb.acl-status |
String |
This parameter is mandatory when you configure an IP address blocklist or trustlist for a load balancer. Options:
|
v1.23.12-r0, v1.25.7-r0, v1.27.4-r0, v1.28.2-r0, or later |
kubernetes.io/elb.acl-type |
String |
This parameter is mandatory when you configure an IP address blocklist or trustlist for a load balancer. Options:
|
v1.23.12-r0, v1.25.7-r0, v1.27.4-r0, v1.28.2-r0, or later |
apiVersion: v1 kind: Service metadata: name: nginx annotations: kubernetes.io/elb.id: <your_elb_id> # Load balancer ID. Replace it with the actual value. kubernetes.io/elb.class: performance # Load balancer type kubernetes.io/elb.acl-id: <your_acl_id> # ID of an IP address group for accessing a load balancer kubernetes.io/elb.acl-status: 'on' # Enable access control. kubernetes.io/elb.acl-type: 'white' # Trustlist for access control spec: selector: app: nginx ports: - name: service0 port: 80 protocol: TCP targetPort: 80 type: LoadBalancer
Host Network
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/hws-hostNetwork |
String |
If the pod uses hostNetwork, the ELB forwards the request to the host network after this annotation is used. Options:
|
v1.9 or later |
apiVersion: v1 kind: Service metadata: name: nginx annotations: kubernetes.io/elb.id: <your_elb_id> # Load balancer ID. Replace it with the actual value. kubernetes.io/elb.class: performance # Load balancer type kubernetes.io/hws-hostNetwork: 'true' # The load balancer forwards the request to the host network. spec: selector: app: nginx ports: - name: service0 port: 80 protocol: TCP targetPort: 80 type: LoadBalancer
Timeout
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.keepalive_timeout |
String |
Timeout for client connections. If there are no requests reaching the load balancer during the timeout duration, the load balancer will disconnect the connection from the client and establish a new connection when there is a new request. Value:
|
Dedicated load balancers: v1.19.16-r30, v1.21.10-r10, v1.23.8-r10, v1.25.3-r10, or later Shared load balancers: v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later |
For details, see Configuring Timeout for a LoadBalancer Service.
Resource Tags
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.tags |
String |
Add resource tags to a load balancer. This parameter can be configured only when a load balancer is automatically created. A tag is in the format of "key=value". Use commas (,) to separate multiple tags. |
v1.23.11-r0, v1.25.6-r0, v1.27.3-r0, or later |
For details, see Using kubectl to Create a Service (Automatically Creating a Load Balancer).
HTTP/2
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.http2-enable |
String |
Whether HTTP/2 is enabled. Request forwarding using HTTP/2 improves the access performance between your application and the load balancer. However, the load balancer still uses HTTP/1.x to forward requests to the backend server. Options:
Note: HTTP/2 can be enabled or disabled only when the listener uses HTTPS. This parameter is invalid and defaults to false when the listener protocol is HTTP. |
v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later |
For details, see Configuring HTTP/2 for a LoadBalancer Service.
Enabling GZIP
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.gzip-enabled |
String |
LoadBalancer Services support data compression, which reduces the size of files to be transferred, improves file transfer efficiency, and reduces the bandwidth needed for the transmission. If this function is enabled, specific files will be compressed. If you do not enable this function, files will not be compressed. By default, data compression is disabled. The files in the following format can be compressed:
This function is available only for HTTP/HTTPS listeners of dedicated load balancers. If the advanced configuration for enabling data compression or the target annotation is deleted, the ELB configuration will not be modified. |
v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later |
For details, see Configuring GZIP Data Compression for a LoadBalancer Service.
Configuring Graceful Exit for the Load Balancer Backend
You are allowed to configure graceful exit time for the backend servers of load balancers associated with Services providing layer-4 load balancing for clusters. The default setting for graceful exit time is enabled and set to 300s.
ELB does not offer this feature to everyone. If you want to use it, you need to submit a service ticket to ELB.
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.connection-drain-enable |
String |
Specifies whether to enable graceful exit for the load balancer backend. |
v1.23.13-r0, v1.25.8-r0, v1.27.5-r0, v1.28.3-r0, or later |
kubernetes.io/elb.connection-drain-timeout |
String |
Specifies the graceful exit time of the load balancer backend. |
Obtaining Client IP Addresses
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.transparent-client-ip |
String |
This parameter is available only when a shared load balancer is used to create a LoadBalancer Service that is TCP/UDP-compliant.
|
v1.23.17-r0, v1.25.12-r0, v1.27.9-r0, v1.28.7-r0, v1.29.3-r0, or later |
For details, see Enabling a LoadBalancer Service to Obtain the Client IP Address.
Configuring a Custom EIP
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.custom-eip-id |
String |
ID of the custom EIP, which can be seen on the EIP console The EIP must be bindable. |
v1.23.18-r0, v1.25.13-r0, v1.27.10-r0, v1.28.8-r0, v1.29.4-r0, v1.30.1-r0, and later versions |
For details, see Configuring a Custom EIP for a LoadBalancer Service.
Configuring Port Ranges for a Listener
Parameter |
Type |
Description |
Supported Cluster Version |
---|---|---|---|
kubernetes.io/elb.port-ranges |
String |
If a dedicated load balancer is used and the TCP, UDP, or TLS protocol is selected, you can create a listener that listens to ports within a certain range from 1 to 65535. You can add a maximum of 10 port ranges that do not overlap for each listener. The parameter value is in the following format, where ports_name and port must be unique: '{"<ports_name_1>":["<port_1>,<port_2>","<port_3>,<port_4>"], "<ports_name_2>":["<port_5>,<port_6>","<port_7>,<port_8>"]}' For example, the port names are cce-service-0 and cce-service-1, and the listener listens to ports 100–200 and 300–400, and 500–600 and 700–800, respectively. '{"cce-service-0":["100,200", "300,400"], "cce-service-1":["500,600", "700,800"]}'
NOTE:
This function relies on ELB capabilities. Before using this function, check whether it is supported in the current region. For details about the regions where this function is supported, see Elastic Load Balance Function Overview. |
v1.23.18-r0, v1.25.13-r0, v1.27.10-r0, v1.28.8-r0, v1.29.4-r0, v1.30.1-r0, and later versions |
For details, see Configuring a Range of Listening Ports for LoadBalancer Services.
Parameters for Automatically Creating a Load Balancer
Parameter |
Mandatory |
Type |
Description |
---|---|---|---|
name |
No |
String |
Name of the automatically created load balancer. The value can contain 1 to 64 characters. Only letters, digits, underscores (_), hyphens (-), and periods (.) are allowed. Default: cce-lb+service.UID |
type |
No |
String |
Network type of the load balancer.
Default: inner |
bandwidth_name |
Yes for public network load balancers |
String |
Bandwidth name. The default value is cce-bandwidth-******. The value can contain 1 to 64 characters. Only letters, digits, underscores (_), hyphens (-), and periods (.) are allowed. |
bandwidth_chargemode |
No |
String |
Bandwidth billing mode.
Default: bandwidth |
bandwidth_size |
Yes for public network load balancers |
Integer |
Bandwidth size. The value ranges from 1 Mbit/s to 2000 Mbit/s by default. Configure this parameter based on the bandwidth range allowed in your region.
The minimum increment for bandwidth adjustment varies depending on the bandwidth range.
|
bandwidth_sharetype |
Yes for public network load balancers |
String |
Bandwidth sharing mode.
|
eip_type |
Yes for public network load balancers |
String |
EIP type.
The specific type varies with regions. For details, see the EIP console. |
vip_subnet_cidr_id |
No |
String |
Subnet where a load balancer is located. The subnet must belong to the VPC where the cluster resides. If this parameter is not specified, the ELB load balancer and the cluster are in the same subnet. This field can be specified only for clusters of v1.21 or later. |
vip_address |
No |
String |
Private IP address of the load balancer. Only IPv4 addresses are supported. The IP address must be in the ELB CIDR block. If this parameter is not specified, an IP address will be automatically assigned from the ELB CIDR block. This parameter is available only in clusters of v1.23.11-r0, v1.25.6-r0, v1.27.3-r0, or later versions. |
available_zone |
Yes |
Array of strings |
AZ where the load balancer is located. You can obtain all supported AZs by getting the AZ list. This parameter is available only for dedicated load balancers. |
l4_flavor_name |
Yes |
String |
Flavor name of the layer-4 load balancer. You can obtain all supported types by getting the flavor list. This parameter is available only for dedicated load balancers. |
l7_flavor_name |
No |
String |
Flavor name of the layer-7 load balancer. You can obtain all supported types by getting the flavor list. This parameter is available only for dedicated load balancers. The value of this parameter must be the same as that of l4_flavor_name, that is, both are elastic specifications or fixed specifications. |
elb_virsubnet_ids |
No |
Array of strings |
Subnet where the backend server of the load balancer is located. If this parameter is left blank, the default cluster subnet is used. Load balancers occupy different number of subnet IP addresses based on their specifications. Do not use the subnet CIDR blocks of other resources (such as clusters and nodes) as the load balancer CIDR block. This parameter is available only for dedicated load balancers. Example: "elb_virsubnet_ids": [ "14567f27-8ae4-42b8-ae47-9f847a4690dd" ] |
ipv6_vip_virsubnet_id |
No |
String |
ID of the IPv6 subnet where the load balancer resides. IPv6 must be enabled for the corresponding subnet. This parameter is mandatory only when the dual-stack clusters are used. This parameter is available only for dedicated load balancers. |
Parameter |
Mandatory |
Type |
Description |
---|---|---|---|
delay |
No |
String |
Health check interval (s) Value range: 1 to 50. Default value: 5 |
timeout |
No |
String |
Health check timeout, in seconds. Value range: 1 to 50. Default value: 10 |
max_retries |
No |
String |
Maximum number of health check retries. Value range: 1 to 10. Default value: 3 |
protocol |
No |
String |
Health check protocol. Value options: TCP or HTTP |
path |
No |
String |
Health check URL. This parameter needs to be configured when the protocol is HTTP. Default value: / Value range: 1-80 characters |
Parameter |
Mandatory |
Type |
Description |
---|---|---|---|
target_service_port |
Yes |
String |
Port for health check specified by spec.ports. The value consists of the protocol and port number, for example, TCP:80. |
monitor_port |
No |
String |
Re-specified port for health check. If this parameter is not specified, the service port is used by default.
NOTE:
Ensure that the port is in the listening state on the node where the pod is located. Otherwise, the health check result will be affected. |
delay |
No |
String |
Health check interval (s) Value range: 1 to 50. Default value: 5 |
timeout |
No |
String |
Health check timeout, in seconds. Value range: 1 to 50. Default value: 10 |
max_retries |
No |
String |
Maximum number of health check retries. Value range: 1 to 10. Default value: 3 |
protocol |
No |
String |
Health check protocol. Default value: protocol of the associated Service Value options: TCP, UDP, or HTTP |
path |
No |
String |
Health check URL. This parameter needs to be configured when the protocol is HTTP. Default value: / Value range: 1-80 characters |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot