Configuring Domain Name Resolution for CCE Containers

Service

• Create a Service before you create a workload (Deployment or ReplicaSet). When the Kubernetes starts a container, it provides environment variables that point to all the Services that are running when the container is started. For example, if a Service named foo exists, all containers will obtain the following variables when they are initialized.

  FOO_SERVICE_HOST=<the host the Service is running on>
  FOO_SERVICE_PORT=<the port the Service is running on>

  Therefore, you must create a Service first. Otherwise, the environment variables do not take effect. This restriction does not apply to DNS.

• CCE clusters provide the coredns add-on as the DNS server. The DNS server monitors the Kubernetes APIs for the new Services and creates a set of DNS records for each Service. If DNS is enabled throughout the cluster, all pods will be able to automatically resolve the names of Services.
• Do not specify a hostPort for a pod unless necessary. When a pod is bound to a hostPort, the number of locations to which the pod can be scheduled will be limited because each <hostIP, hostPort, protocol> must be unique. If you do not specify hostIP and protocol, Kubernetes uses 0.0.0.0 as the default host IP address and TCP as the default protocol.

If you only need to access the port for debugging, you can use apiserver proxies or kubectl port-forward.

If you want to open the pod port on the node, consider using the NodePort Service before using hostPort.

• Do not use hostNetwork. The reason is the same as that of using hostPort.
• When kube-proxy load balancing is not required, use headless Services (ClusterIP set to None) for service discovery.

DNS

By default, CCE provides a DNS add-on Service named coredns to automatically assign DNS domain names for other Services. If it is running in the cluster, run the following command to check the status:

kubectl get services coredns --namespace=kube-system
NAME       TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE
kube-dns   ClusterIP   10.0.0.10    <none>        53/UDP,53/TCP   8m

If the pod is not running, you can run the describe command to check why the pod is not started. Assume that there is a Service that has a permanent IP address and a DNS server (coredns cluster add-on) that assigns domain name to the IP address. In this way, any pod in the cluster can communicate with the Service. You can run another application for testing. Enable a new pod, access the pod, and run the curl command to check whether the domain name of the Service can be correctly resolved. In some cases, the curl command cannot be executed due to the DNS search principles and configuration.

When a pod is created on the CCE console, not all dnsConfig configurations are opened and some default values of the pod domain name resolution parameters are used. You need to know well the default configurations. A typical case is ndots. If the number of dots is within the ndots threshold range, the domain name is considered as an internal domain name of the Kubernetes cluster and the ..svc.cluster.local suffix is added to the domain name.

DNS Search Principles and Rules

DNS configuration file: /etc/resolv.conf

nameserver 10.247.x.x
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:3

Parameters:

• nameserver: domain name resolution server
• search: domain name suffix search rule. More search configurations indicate more matching times for domain name resolution. For example, if three suffixes are matched, at least six search operations are required because both IPv4 and IPv6 addresses need to be checked.
• options: domain name resolution option. Multiple KV values are available. A typical case is ndots. If the number of dots in the domain name to be accessed exceeds the value of ndots, the domain name is considered as a complete domain name and is directly parsed. If the number of dots is less than the value of ndots, the suffix ..svc.cluster.local will be added.

Parameters in Kubernetes dnsConfig

• nameservers: a list of IP addresses that will be used as DNS servers for the pod. A maximum of three IP addresses can be specified. If pod's dnsPolicy is set to None, the list must contain at least one IP address, otherwise this property is optional. The servers listed will be combined to the base nameservers generated from the specified DNS policy with duplicate addresses removed.
• searches: a list of DNS search domains for hostname lookup in the pod. This property is optional. When specified, the provided list will be merged into the base search domain names generated from the chosen DNS policy. Duplicate domain names are removed. Kubernetes allows for at most 6 search domains.
• options: an optional list of objects where each object may have a name property (required) and a value property (optional). The contents in this property will be merged to the options generated from the specified DNS policy. Duplicate entries are removed.

For details, see DNS for Services and Pods.

Pod DNS Policies

DNS policies can be set on a per-pod basis. Currently, three types of DNS policies are supported: Default, ClusterFirst, and None.

• Default: The DNS configuration of the pod inherits from the host. That is, the DNS configuration of the pod is the same as that of the host and node.
• ClusterFirst: Unlike the Default policy, the ClusterFirst policy writes kube-dns (or CoreDNS) information to the DNS configuration of the pod in advance. ClusterFirst is the default pod policy. If PodPolicy is not specified for the pod, dnsPolicy is preset to ClusterFirst. However, ClusterFirst is mutually exclusive with HostNetwork=true. If HostNetwork is set to true, the ClusterFirst policy will be forcibly changed to the Default policy.
• None: This policy will clear the DNS configuration preset for the pod. If dnsPolicy is set to None, Kubernetes does not load any DNS configuration that is determined by its own logic in advance for the pod. Therefore, if you want to set dnsPolicy to None, you are advised to set dnsConfig to describe custom DNS parameters. This setting ensures that the pod has DNS configuration.

DNS configuration scenarios are provided as follows:

Scenario 1: Using a custom DNS

The following example allows you to use a custom DNS to resolve the application domain name configuration in pods. After application migration, you do not need to modify the configuration.

apiVersion: v1
kind: Pod
metadata:
  namespace: default
  name: dns-example
spec:
  containers:
    - name: test
      image: nginx
  dnsPolicy: "None"
  dnsConfig:
    nameservers:
      - 1.2.3.4
    searches:
      - ns1.svc.cluster.local
      - my.dns.search.suffix
    options:
      - name: ndots
        value: "2"
      - name: edns0

Scenario 2: Using the Kubernetes DNS add-on CoreDNS

The DNS service of Kubernetes is preferentially used for domain name resolution. If the resolution fails, the DNS service of an external cascading system is used for domain name resolution.

apiVersion: v1
kind: Pod
metadata:
  namespace: default
  name: dns-example
spec:
  containers:
    - name: test
      image: nginx
  dnsPolicy: ClusterFirst

Scenario 3: Using the public network domain name resolution

This mode applies to the scenario where the domain names in pods are to be accessed from public networks. In this case, the applications in the pods resolve domain names from an external DNS.

apiVersion: v1
kind: Pod
metadata:
  namespace: default
  name: dns-example
spec:
  containers:
    - name: test
      image: nginx
  dnsPolicy: Default

Scenario 4: Using hostNetwork

If hostNetwork: true is used to configure the networking in the pod, the network ports of the host machine are exposed to the application running in the pod. All network ports on the LAN where the host machine is located can be used to access the application.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx
spec:
  template:
    metadata:
      labels:
        app: nginx
    spec:
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80

If dnsPolicy: ClusterFirstWithHostNet is not added, even if the pod uses the DNS of the host machine by default, other pods in the Kubernetes cluster cannot be accessed through the Service name in the container.

CoreDNS Configuration

1. Configuring the CoreDNS ConfigMap