Using kubectl to Connect to a Federation

Permissions

When you use kubectl to connect to a federation, UCS uses kubeconfig.json generated on the federation for authentication. This file contains user information, based on which UCS determines which Kubernetes resources can be accessed by kubectl. The permissions recorded in a kubeconfig.json file vary from user to user.

Constraints

• For security purposes, the federation API server does not have a public IP address. UCS creates an endpoint in your VPC and subnet and connects the endpoint to the federation API server for the access to the federation. For each federation, only one endpoint is created in the same VPC. If a VPC already has an endpoint for connecting to the federation API server, the endpoint will be reused.
• Currently, the kubectl configuration file can be downloaded only for projects in AP-Singapore.

Prerequisites

• Before using kubectl to connect to a federation, ensure that the federation has been enabled (Enabling Cluster Federation) and is running normally.
• Only the client in a VPC can connect to a federation using kubectl. If there is no client in the VPC, create one.
• kubectl has been downloaded and uploaded to the client. For details about how to download kubectl, see Kubernetes releases.
• At least the custom policy iam:clustergroups:get has been created.

Using kubectl to Connect to a Federation

1. Log in to the UCS console and click the fleet name to access the fleet console. Then, click kubectl in Fleet Info.
   Figure 1 kubectl connection
   

2. Select a project, VPC, master node subnet, and validity period as prompted and click Download to download the kubectl configuration file.
   The name of the downloaded file is {Fleet name}_kubeconfig.json.

   Figure 2 Using kubectl to connect to a federation instance
   
   

   • If the {Fleet name}_kubeconfig.json file is leaked, your clusters may be attacked. Keep it secure.
   • The validity period of the kubectl configuration file can be set as required. The options are 5 years, 1 year, 6 months, 30 days, and 15 days to 1 day. The minimum value is 1 day.

3. Install and configure kubectl on the executor.
   1. Copy kubectl and its configuration file to the /home directory on the executor in the selected VPC and subnet.
   2. Log in to your executor and configure kubectl.

      cd /home
      chmod +x kubectl
      mv -f kubectl /usr/local/bin
      mkdir -p $HOME/.kube
      mv -f <fleet-name>_kubeconfig.json $HOME/.kube/config   --Change the fleet name in the command to the actual fleet name.

Resources and Operations Supported by a Federation

Table 1 lists the resources and operations supported by a federation. In the table, "√" means the operation can be performed on related resources. "Partially supported" means the operation can be performed on part of the resources. If there is neither "√" nor "partially supported", the operation cannot be performed on related resources.

• Custom resources in a cluster can be operated through the federation entry only after the CRD is registered with the federation.
• The UPDATE and PATCH operations of ingress objects can be only performed on resources on the federation control plane, not in member clusters.

FAQ

• If the following error message is displayed when you access federated resources, you do not have permissions to operate the resources. In this case, apply for permissions by referring to RBAC Authorization for Cluster Federations.

  

• If "Precondition Required" is displayed when you access the federation and cluster resources, the cluster may be disconnected due to network problems or member cluster faults. Perform the following steps to locate the fault.

  

  1. Check whether the member cluster is running normally and whether it can be connected to UCS. For details, see What Can I Do If an Attached Cluster Fails to Be Connected?
  2. Check whether the proxy-agent deployed in the member cluster is running normally. For details, see What Can I Do If proxy-agent Fails to Be Deployed?

• If "no such host" is displayed, perform the following steps to locate the fault.

  

  1. Check whether the VPC endpoint exists or is deleted accidentally.

     server=`cat config  | jq '.clusters[0].cluster.server'` echo ${server:15:36}

  2. If the VPC endpoint exists, check whether the executor and the VPC endpoint are in the same VPC and whether the network connection between them is normal.

• If "You must be logged in to the server (Unauthorized)" is displayed, perform the following steps to locate the fault.
  

  1. Check whether the certificates are correct.
     Save the certificates to a temporary file.

     cd ~/.kube
     cat config  | jq '.clusters[0].cluster."certificate-authority-data"' | tr -d '"' | base64 -d > ca.crt
     cat config  | jq '.users[0].user."client-certificate-data"' | tr -d '"' | base64 -d > tls.crt
     cat config  | jq '.users[0].user."client-key-data"' | tr -d '"' | base64 -d > tls.key

     Check whether the TLS certificate and the CA certificate match.

     openssl verify -CAfile ca.crt tls.crt

     If "tls.crt: OK" is displayed, the CA certificate is correct. If it is not displayed, download the kubeconfig file again.

     

     If the certificate verification command fails to be executed and a message is displayed indicating that there is no jq, check whether jq is installed on the node.

  2. Check whether the public key and private key of the certificate match.

     diff -eq <(openssl x509 -pubkey -noout -in tls.crt) <(openssl rsa -pubout -in tls.key)

     If "writing RSA key" is displayed, the public key and private key match. If they do not match, download the kubeconfig file again. After the verification is complete, delete the temporary file.

     rm -f ca.crt tls.crt tls.key

  3. Check whether the certificate has expired.
     Save the certificate to a temporary file.

     cd ~/.kube
     cat config  | jq '.users[0].user."client-certificate-data"' | tr -d '"' | base64 -d > tls.crt

     Check the certificate validity period.

     openssl x509 -noout -text -in tls.crt  | grep -E "Not Before|Not After"

     The certificate validity period is shown in the following figure. Check whether the current certificate is within the validity period. If the certificate expires, download the kubeconfig file again and delete the temporary file.

     rm -f tls.crt