Updated on 2024-11-12 GMT+08:00

Configuring an HTTPS Certificate for a LoadBalancer Ingress

Ingresses support SSL or TLS certificates, allowing you to secure your Services with HTTPS.

You are allowed to use either of the following ways to configure an ingress certificate in a cluster:
  • Using a TLS Certificate: You need to first import a certificate to a Secret. CCE will then automatically handle the certificate configurations on the ELB console and give a name to the certificate (started with k8s_plb_default). This certificate, which is generated by CCE, cannot be modified or deleted from the ELB console. To modify the certificate, update the secret where the certificate is imported to on CCE.
  • Using a Certificate Created on the ELB Console: You are allowed to directly use certificates created on the ELB console. There is no need to manually configure the cluster Secrets, and you can modify the certificates on the ELB console.

If HTTPS is enabled for the same port of the same load balancer of multiple ingresses, you must select the same certificate. For details, see Configuring a Same Port for Multiple Ingresses.

Prerequisites

Using a TLS Certificate

You can configure a TLS certificate using either the CCE console or kubectl.

  1. Log in to the CCE console and click the cluster name to access the cluster console.
  2. Choose Services & Ingresses in the navigation pane, click the Ingresses tab, and click Create Ingress in the upper right corner.
  3. Configure ingress parameters.

    This example explains only key parameters for configuring HTTPS certificates. You can configure other parameters as required. For details, see Creating a LoadBalancer Ingress on the Console.

    Table 1 Key parameters

    Parameter

    Description

    Example

    Name

    Enter an ingress name.

    ingress-test

    Load Balancer

    Select a load balancer to be associated with the ingress or automatically create a load balancer.

    Shared

    Listener

    • External Protocol: Select HTTPS when configuring a certificate for an ingress.
    • External Port: specifies the port of the load balancer listener. The default HTTPS port is 443.
    • Certificate Source: Select TLS secret.
    • Server Certificate: kubernetes.io/tls and IngressTLS are supported.

      If no certificate is available, you can create a TLS certificate. For details about the configuration parameters, see Creating a Secret.

    • External Protocol: HTTPS
    • External Port: 443
    • Certificate Source: TLS secret
    • Server Certificate: test

    Forwarding Policy

    • Domain Name: Enter an actual domain name to be accessed. If it is left blank, the ingress can be accessed through the IP address. Ensure that the domain name has been registered and licensed. Once a forwarding policy is configured with a domain name specified, you must use the domain name for access.
    • Path Matching Rule: Select Prefix match, Exact match, or RegEx match.
    • Path: Enter the path provided by a backend application for external access. The path added must be valid in the backend application, or the forwarding cannot take effect.
    • Destination Service: Select an existing Service or create a Service. Any Services that do not match the search criteria will be filtered out automatically.
    • Destination Service Port: Select the access port of the destination Service.
    • Domain Name: You do not need to configure this parameter.
    • Path Matching Rule: Prefix match
    • Path: /
    • Destination Service: nginx
    • Destination Service Port: 80
    Figure 1 Selecting a TLS certificate

  4. Click OK.
  1. Use kubectl to access the cluster. For details, see Connecting to a Cluster Using kubectl.
  2. Ingress supports two TLS secret types: kubernetes.io/tls and IngressTLS. IngressTLS is used as an example. For details, see Creating a Secret. For details about examples of the kubernetes.io/tls secret and its description, see TLS secrets.

    Create a YAML file named ingress-test-secret.yaml. The file name can be customized.

    vi ingress-test-secret.yaml
    The YAML file is configured as follows:
    apiVersion: v1
    data:
      tls.crt: LS0******tLS0tCg==
      tls.key: LS0tL******0tLS0K
    kind: Secret
    metadata:
      annotations:
        description: test for ingressTLS secrets
      name: ingress-test-secret
      namespace: default
    type: IngressTLS

    In the preceding information, tls.crt and tls.key are only examples. Replace them with the actual files. The values of tls.crt and tls.key are Base64-encoded.

  3. Create a secret.

    kubectl create -f ingress-test-secret.yaml

    If information similar to the following is displayed, the secret has been created:

    secret/ingress-test-secret created

  4. Check the created secret.

    kubectl get secrets

    If information similar to the following is displayed, the secret has been created:

    NAME                         TYPE                                  DATA      AGE
    ingress-test-secret          IngressTLS                            2         13s

  5. Create a YAML file named ingress-test.yaml. The file name can be customized.

    vi ingress-test.yaml

    Default security policy (kubernetes.io/elb.tls-ciphers-policy) is supported only in clusters of v1.17.17 or later.

    Custom security policy (kubernetes.io/elb.security_policy_id) is supported only in clusters of v1.17.17 or later.

    An example YAML file of an ingress associated with an automatically created load balancer is as follows:

    For clusters of v1.21 or earlier:

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress 
    metadata: 
      name: ingress-test
      annotations: 
        kubernetes.io/elb.class: performance
        kubernetes.io/ingress.class: cce
        kubernetes.io/elb.port: '443'
        kubernetes.io/elb.autocreate: 
          '{
              "type": "public",
              "bandwidth_name": "cce-bandwidth-******",
              "bandwidth_chargemode": "bandwidth",
              "bandwidth_size": 5,
              "bandwidth_sharetype": "PER",
              "eip_type": "5_bgp",
              "available_zone": [
                  "ap-southeast-1a"
              ],
              "elb_virsubnet_ids":["b4bf8152-6c36-4c3b-9f74-2229f8e640c9"],
              "l7_flavor_name": "L7_flavor.elb.s1.small"
           }'
        kubernetes.io/elb.security_policy_id: 99bec42b-0dd4-4583-98e9-b05ce628d157  # The priority of the custom security policy is higher than that of the default security policy.
        kubernetes.io/elb.tls-ciphers-policy: tls-1-2
    spec:
      tls: 
      - secretName: ingress-test-secret
      rules: 
      - host: ''
        http: 
          paths: 
          - path: '/'
            backend: 
              serviceName: <your_service_name>  # Replace it with the name of your target Service.
              servicePort: 80
            property:
              ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH
    For clusters of v1.23 or later:
    apiVersion: networking.k8s.io/v1
    kind: Ingress 
    metadata: 
      name: ingress-test
      annotations: 
        kubernetes.io/elb.class: performance
        kubernetes.io/elb.port: '443'
        kubernetes.io/elb.autocreate: 
          '{
              "type": "public",
              "bandwidth_name": "cce-bandwidth-******",
              "bandwidth_chargemode": "bandwidth",
              "bandwidth_size": 5,
              "bandwidth_sharetype": "PER",
              "eip_type": "5_bgp",
              "available_zone": [
                  "ap-southeast-1a"
              ],
              "elb_virsubnet_ids":["b4bf8152-6c36-4c3b-9f74-2229f8e640c9"],
              "l7_flavor_name": "L7_flavor.elb.s1.small"
           }'
        kubernetes.io/elb.security_policy_id: 99bec42b-0dd4-4583-98e9-b05ce628d157  # The priority of the custom security policy is higher than that of the default security policy.
        kubernetes.io/elb.tls-ciphers-policy: tls-1-2
    spec:
      tls: 
      - secretName: ingress-test-secret
      rules: 
      - host: ''
        http: 
          paths: 
          - path: '/'
            backend: 
              service:
                name: <your_service_name>  # Replace it with the name of your target Service.
                port: 
                  number: 80             # Replace 80 with the port number of your target Service.
            property:
              ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH
            pathType: ImplementationSpecific
      ingressClassName: cce 
    Table 2 Key parameters

    Parameter

    Mandatory

    Type

    Description

    kubernetes.io/elb.security_policy_id

    No

    String

    The ID of the custom security group policy in ELB. Obtain it on the ELB console. This field takes effect only when HTTPS is used and has a higher priority than the default security policy.

    For details about how to create and update a custom security policy, see TLS Security Policy.

    kubernetes.io/elb.tls-ciphers-policy

    No

    String

    The default value is tls-1-2, which is the default security policy used by the listener and takes effect only when HTTPS is used.

    Options:

    • tls-1-0
    • tls-1-1
    • tls-1-2
    • tls-1-2-strict

    For details of cipher suites for each security policy, see Table 3.

    tls

    No

    Array of strings

    When HTTPS is used, this parameter must be added to specify the secret certificate.

    Multiple independent domain names and certificates can be added. For details, see Configuring SNI for a LoadBalancer Ingress.

    secretName

    No

    String

    This parameter is mandatory if HTTPS is used. Set this parameter to the name of the created secret.

    Table 3 tls_ciphers_policy parameter description

    Security Policy

    TLS Version

    Cipher Suite

    tls-1-0

    TLS 1.2

    TLS 1.1

    TLS 1.0

    ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:AES128-SHA256:AES256-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128-SHA:AES256-SHA

    tls-1-1

    TLS 1.2

    TLS 1.1

    tls-1-2

    TLS 1.2

    tls-1-2-strict

    TLS 1.2

    ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:AES128-SHA256:AES256-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384

    TLS-1-0-WITH-1-3 (dedicated load balancer)

    TLS 1.3

    TLS 1.2

    TLS 1.1

    TLS 1.0

    ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:AES128-SHA256:AES256-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128-SHA:AES256-SHA:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_8_SHA256:TLS_AES_128_CCM_SHA256

    TLS-1-2-FS (dedicated load balancer)

    TLS 1.3

    TLS 1.2

    ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384

    TLS-1-2-FS-WITH-1-3 (dedicated load balancer)

    TLS 1.3

    TLS 1.2

    ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_8_SHA256:TLS_AES_128_CCM_SHA256

  6. Create an ingress.

    kubectl create -f ingress-test.yaml

    If information similar to the following is displayed, the ingress has been created:

    ingress/ingress-test created

  7. Check the created ingress.

    kubectl get ingress

    If information similar to the following is displayed, the ingress has been created:

    NAME          CLASS    HOSTS     ADDRESS          PORTS   AGE
    ingress-test  cce      *         121.**.**.**     80,443  10s

  8. Enter https://121.**.**.**:443 in the address box of the browser to access the workload (for example, Nginx workload).

    121.**.**.** indicates the IP address of the unified load balancer.

Using a Certificate Created on the ELB Console

You can configure a certificate created on the ELB console for an ingress using either the CCE console or kubectl.

  • If both an ELB certificate and a TLS certificate are specified for the same ingress, the ingress will use the ELB certificate.
  • CCE does not check whether an ELB certificate is valid. It only checks whether the certificate is present.
  • Only ingresses in clusters of v1.19.16-r2, v1.21.5-r0, v1.23.3-r0, or later support ELB certificates.
  1. Log in to the CCE console and click the cluster name to access the cluster console.
  2. Choose Services & Ingresses in the navigation pane, click the Ingresses tab, and click Create Ingress in the upper right corner.
  3. Configure ingress parameters.

    This example explains only key parameters for configuring HTTPS certificates. You can configure other parameters as required. For details, see Creating a LoadBalancer Ingress on the Console.

    Table 4 Key parameters

    Parameter

    Description

    Example

    Name

    Enter an ingress name.

    ingress-test

    Load Balancer

    Select a load balancer to be associated with the ingress or automatically create a load balancer.

    Shared

    Listener

    • External Protocol: Select HTTPS.
    • External Port: specifies the port of the load balancer listener. The default HTTPS port is 443.
    • Certificate Source: Select ELB server certificate.
    • Server Certificate: Use a certificate created on ELB.

      If no certificate is available, go to the ELB console and create one. For details, see Adding a Certificate.

    • External Protocol: HTTPS
    • External Port: 443
    • Certificate Source: ELB server certificate
    • Server Certificate: cert-test

    Forwarding Policy

    • Domain Name: Enter an actual domain name to be accessed. If it is left blank, the ingress can be accessed through the IP address. Ensure that the domain name has been registered and licensed. Once a forwarding policy is configured with a domain name specified, you must use the domain name for access.
    • Path Matching Rule: Select Prefix match, Exact match, or RegEx match.
    • Path: Enter the path provided by a backend application for external access. The path added must be valid in the backend application, or the forwarding cannot take effect.
    • Destination Service: Select an existing Service or create a Service. Any Services that do not match the search criteria will be filtered out automatically.
    • Destination Service Port: Select the access port of the destination Service.
    • Domain Name: You do not need to configure this parameter.
    • Path Matching Rule: Prefix match
    • Path: /
    • Destination Service: nginx
    • Destination Service Port: 80
    Figure 2 Selecting a certificate from the ELB service

  4. Click OK.

To use an ELB certificate, you can specify the kubernetes.io/elb.tls-certificate-ids annotation.

  1. Use kubectl to access the cluster. For details, see Connecting to a Cluster Using kubectl.
  2. Create a YAML file named ingress-test.yaml. The file name can be customized.

    vi ingress-test.yaml

    An example YAML file of an ingress associated with an existing load balancer is as follows:

    For clusters of v1.21 or earlier:

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress 
    metadata: 
      name: ingress-test
      annotations: 
        kubernetes.io/ingress.class: cce
        kubernetes.io/elb.port: '443'
        kubernetes.io/elb.id: 0b9a6c4d-bd8b-45cc-bfc8-ff0f9da54e95
        kubernetes.io/elb.class: union
        kubernetes.io/elb.tls-certificate-ids: 058cc023690d48a3867ad69dbe9cd6e5,b98382b1f01c473286653afd1ed9ab63
    spec:
      rules: 
      - host: ''
        http: 
          paths: 
          - path: '/'
            backend: 
              serviceName: <your_service_name>  # Replace it with the name of your target Service.
              servicePort: 80
            property:
              ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH
    For clusters of v1.23 or later:
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: ingress-test
      namespace: default
      annotations:
        kubernetes.io/elb.port: '443'
        kubernetes.io/elb.id: 0b9a6c4d-bd8b-45cc-bfc8-ff0f9da54e95
        kubernetes.io/elb.class: union
        kubernetes.io/elb.tls-certificate-ids: 058cc023690d48a3867ad69dbe9cd6e5,b98382b1f01c473286653afd1ed9ab63
    spec:
      rules:
        - host: ''
          http:
            paths:
              - path: '/'
                backend:
                  service:
                    name: <your_service_name>  # Replace it with the name of your target Service.
                    port: 
                      number: 80             # Replace 80 with the port number of your target Service.
                property:
                  ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH
                pathType: ImplementationSpecific
      ingressClassName: cce
    Table 5 Key parameters

    Parameter

    Type

    Description

    kubernetes.io/elb.tls-certificate-ids

    String

    ELB certificate IDs, which are separated by comma (,). The list length is greater than or equal to 1. The first ID in the list is the server certificate, and the other IDs are SNI certificates in which a domain name must be contained.

    If an SNI certificate cannot be found based on the domain name requested by the client, the server certificate will be returned by default.

    To obtain the certificate, log in to the CCE console, choose Service List > Networking > Elastic Load Balance, and click Certificates in the navigation pane. In the load balancer list, copy the ID under the target certificate name.

  3. Create an ingress.

    kubectl create -f ingress-test.yaml

    If information similar to the following is displayed, the ingress has been created:

    ingress/ingress-test created

  4. Check the created ingress.

    kubectl get ingress

    If information similar to the following is displayed, the ingress has been created:

    NAME          CLASS    HOSTS     ADDRESS          PORTS   AGE
    ingress-test  cce      *         121.**.**.**     80,443  10s

Configuring a Same Port for Multiple Ingresses

In a cluster, multiple ingresses can share a listener, allowing them to use the same port on a single load balancer. When two ingresses are set up with HTTPS certificates, the server certificate that is used will be based on the configuration of the earliest ingress.

Starting from v1.21.15-r0, v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, v1.29.1-r0, and later, the YAML file of ingresses includes annotation: kubernetes.io/elb.listener-master-ingress. This annotation specifies the server certificate configured and used by ingresses.

For example, the configurations of two ingresses are as follows:

Ingress Name

ingress1

ingress2

Namespace

default

default

Creation Time

2024-04-01

2024-04-02

Protocol

HTTPS

HTTPS

Load Balancer

elb1

elb1

Port

443

443

kubernetes.io/elb.listener-master-ingress

default/ingress1

default/ingress1

In the configurations of the two ingresses, annotation: kubernetes.io/elb.listener-master-ingress is present and the value is default/ingress1, indicating that the server certificates that take effect for the two ingresses are the server certificates configured for ingress1 in the default namespace.

If ingresses in separate namespaces use the same listener and TLS certificates, the secrets associated with the TLS certificates may not display normal for the later-created ingress due to namespace isolation. For details, see How Can I Synchronize Certificates When Multiple Ingresses in Different Namespaces Share a Listener?

To update a server certificate, modify the certificate in the ingress1 configuration within the default namespace. Once the modification is made, the certificates used by both ingress1 and ingress2 will be updated accordingly.

To find the ingress with the active certificate configuration, run the following command:

kubectl get ingress -n ${namespace} ${ingress_name}  -oyaml | grep kubernetes.io/elb.listener-master-ingress