Installing the Agent in a Third-Party Public Network Cluster

Scenario

Install the agent on a third-party cluster that can access the public network. After the configuration is complete, HSS automatically installs the agent on existing cluster nodes, installs the agent on new nodes when the cluster is scaled out, and uninstalls the agent from removed nodes when the cluster is scaled in.

Constraints and Limitations

• Supported cluster orchestration platforms: Kubernetes 1.19 or later
• Supported node OS: Linux
• Node specifications: at least 2 vCPUs, 4 GiB memory, 40 GiB system disk, and 100 GiB data disk

Step 1: Create a VPC

1. Log in to the console and go to the page for Creating a VPC.
2. On the Create VPC page, set parameters for the VPC and subnets as prompted.
   You are advised to set some parameters by referring to Table 1 and retain the default values for other parameters.

   Table 1 Parameters for creating a VPC

   Parameter	Description	Example Value
   Region	Select a region near you to ensure the lowest latency possible.	-
   Name	VPC name. The name: Must contain 1 to 64 characters. Can contain letters, numbers, underscores (_), hyphens (-), and periods (.).	HSS-outside-anp-VPC
   Enterprise Project	Enterprise project to which the VPC belongs. An enterprise project facilitates project-level management and grouping of cloud resources and users. The name of the default project is default.	default
   Name	Subnet name. The name: Must contain 1 to 64 characters. Can contain letters, numbers, underscores (_), hyphens (-), and periods (.).	HSS-outside-subnet

3. Click Create Now. You can view the VPC after it is created.

Step 2: Create a Security Group

1. In the navigation pane on the left, choose Access Control > Security Groups.
2. Click Create Security Group in the upper right corner.
3. Configure security group parameters as prompted.
   You are advised to configure some parameters by referring to Table 2 and configure other parameters based on site requirements.

   Table 2 Parameters for creating a security group

   Parameter	Description	Example Value
   Name	Specify the name of the security group. The name: Must contain 1 to 64 characters. Can contain letters, numbers, underscores (_), hyphens (-), and periods (.).	HSS-outside-anp-secGroups
   Enterprise Project	When creating a security group, you can add the security group to an enterprise project that has been enabled. An enterprise project facilitates project-level management and grouping of cloud resources and users. The default project is default.	default
   Template	A security group template has preconfigured inbound and outbound rules. You can select one based on your service requirements.	All ports open

4. Click Create Now. You can view the security group after it is created.

Step 3: Create an ECS

1. Click in the upper left corner and Compute > Elastic Cloud Server.
2. In the upper right corner, click Buy ECS.
3. Configure ECS parameters as prompted.

   You are advised to configure some parameters by referring to Table 3 and configure other parameters based on site requirements.

   Table 3 Parameters for purchasing an ECS

   Parameter	Description	Example Value
   Billing Mode	ECS billing mode. Yearly/Monthly: Prepaid mode. Yearly/monthly ECSs are billed by the purchased duration specified in the order. Pay-per-use: Postpaid billing mode. You pay as you go and just pay for what you use. Pay-per-use ECSs are billed by the second and settled by the hour. Spot price: Spot pricing is a postpaid billing mode. You pay as you go and just pay for what you use. In Spot pricing billing mode, your purchased ECS is billed at a lower price than that of a pay-per-use ECS with the same specifications. In Spot pricing billing mode, you can select Spot or Spot block for the Spot Type. Spot ECSs and Spot block ECSs are billed by the second and settled by the hour.	Pay-per-use
   Region	Select a region near you to ensure the lowest latency possible.	-
   CPU Architecture	Select a CPU architecture. The value can be x86 or Kunpeng.	x86
   Specification	Select vCPUs and memory, or enter a keyword to search for ECS specifications. Select ECS specifications by instance family and generation from the list.	General-purpose S6, 2 vCPUs, 4 GiB
   Image	An image is an ECS template that contains an OS. It may also contain proprietary software and application software. You can use images to create ECSs.	Public image, CentOS 7.964bit (40 GiB)
   System Disk	Stores the OS of an ECS, and is automatically created and initialized upon ECS creation.	Ultra-high I/O
   Network	VPC allows you to create logically isolated, configurable, and manageable virtual networks for VPCs. You can configure security groups, Virtual Private Network (VPNs), CIDR blocks, and bandwidths in your VPC. ECSs in different VPCs cannot communicate with each other by default.	HSS-outside-anp-VPC (VPC created in Step 1: Create a VPC)
   Security Group	Select an available security group from the drop-down list. You can select multiple security groups for an ECS (no more than five security groups are recommended). The access rules of all the selected security groups apply to the ECS.	HSS-outside-anp-secGroups (Security group created in Step 2: Create a Security Group)
   EIP	An EIP is a static public IP address bound to a cloud server in a VPC. Using the EIP, the cloud server provides services externally.	Auto assign
   ECS Name	This parameter will be set to the initial server name (hostname) in the ECS OS. The name can contain only letters, digits, underscores (_), hyphens (-), and periods (.).	HSS-outside-anp-ECS
   Login Mode	Mode for logging in to the ECS.	Password
   Enterprise Project	When purchasing an ECS, you can add it to an enabled enterprise project. An enterprise project facilitates project-level management and grouping of cloud resources and users. The name of the default project is default.	default

4. Click Create. In the displayed dialog box, click Agree and Create. After the payment is complete, the ECS will be automatically created and started by default.
5. Click Submit. After the payment is complete, the ECS will be automatically created and started by default.

Step 4: Set Up Nginx

1. Log in to the server created in Step 3: Create an ECS.
2. Go to the temp directory.

   cd /temp

3. Run the following command to create the install_nginx.sh file:

   vi install_nginx.sh

4. Press i to enter the editing mode and copy the following content to the install_nginx.sh file:

   #!/bin/bash
   
   yum -y install pcre-devel zlib-devel popt-devel openssl-devel openssl
   wget http://www.nginx.org/download/nginx-1.21.0.tar.gz
   tar zxf nginx-1.21.0.tar.gz -C /usr/src/
   cd /usr/src/nginx-1.21.0/
   useradd -M -s /sbin/nologin nginx
   ./configure \
   --prefix=/usr/local/nginx \
   --user=nginx \
   --group=nginx \
   --with-file-aio \
   --with-http_stub_status_module \
   --with-http_gzip_static_module \
   --with-http_flv_module \
   --with-http_ssl_module \
   --with-stream \
   --with-pcre && make && make install
   ln -s /usr/local/nginx/sbin/nginx /usr/local/sbin/
   nginx

5. Enter ECS, run the following command, and press Enter to exit.

   :wq!

6. Run the following command to install Nginx:

   bash /tmp/install_nginx.sh

7. Run the following command to modify the Nginx configuration file:

   cat <<END >> /usr/local/nginx/conf/nginx.conf
   stream {
     upstream backend_hss_anp {
       server {{ANP_proxy_address}}:8091 weight=5 max_fails=3 fail_timeout=30s;
     }
     server {
       listen 8091 so_keepalive=on;
       proxy_connect_timeout 10s;
       proxy_timeout 300s;
       proxy_pass backend_hss_anp ;
     }
   }
   END

   Replace {{ANP_proxy_address}} with the actual address and then run the command. For details, see Table 4.

   Table 4 ANP proxy address

   Region	ANP proxy address
   Guiyang1, Bangkok, Shanghai2, Guangzhou, Beijing4, Beijing2, and Shanghai1	hss-proxy.RegionCode.myhuaweicloud.com
   Other	hss-anp.RegionCode.myhuaweicloud.com
   For details about region codes, see Regions and Endpoints.

8. Run the following command to make the Nginx configuration take effect:

   nginx -s reload

9. Run the following command to check whether port 8091 is listened on properly:

   netstat -anp | grep 8091

   If information similar to Figure 1 is displayed, the listening is normal.

   Figure 1 Listening on port 8091 is normal.
   

Step 5: Buy and Configure a ELB

1. Log in to the console and go to the page for Buying ELB page.
2. Set ELB parameters as prompted.
   You are advised to configure some parameters by referring to Table 5 and configure other parameters based on site requirements.

   Table 5 Parameters for buying an ELB

   Parameter	Description	Example Value
   Type	Specifies the type of the shared load balancer. The type cannot be changed after the load balancer is created. Dedicated load balancers work well for heavy-traffic and high-concurrency workloads, such as large websites, cloud native applications, IoV, and multi-AZ disaster recovery applications.	Dedicated
   Billing Mode	Billing mode of a dedicated load balancer. Yearly/Monthly: prepaid billing mode. You pay in advance for a subscription term, and in exchange, you get a discounted rate. Pay-per-use: postpaid billing mode. You pay as you go and just pay for what you use. The load balancer usage is calculated by the second but billed every hour.	Pay-per-use
   Region	Select a region near you to ensure the lowest latency possible.	-
   Name	Specifies the load balancer name. The name can contain: 1 to 64 characters. Letters, digits, underscores (_), hyphens (-), and periods (.).	HSS-outside-anp-ELB
   Enterprise Project	When creating a load balancer, you can add it to an enabled enterprise project. An enterprise project facilitates project-level management and grouping of cloud resources and users. The name of the default project is default.	default
   Specification	Select Elastic or Fixed if pay-per-use is chosen as the billing mode. Specifications: Elastic specifications work well for fluctuating traffic, and you will be charged for how many LCUs you use. Fixed specifications are suitable for stable traffic, and you will be charged for the specifications you select.	Fixed Network load balancing Small
   Network Configuration	Network Type: You can select one or more network types. Public IPv4 network: The load balancer routes IPv4 requests from the clients to backend servers over the Internet. Private IPv4 network: The load balancer routes IPv4 requests from the clients to backend servers in a VPC. If you want the load balancer to route requests from the Internet, bind an EIP to the load balancer. IPv6 network: An IPv6 address will be assigned to the load balancer to route requests from IPv6 clients. VPC: VPC to which the load balancer belongs. After a dedicated load balancer is created, it cannot be changed. Plan the network properly. Select an existing VPC, or click View VPCs to create a desired one. Frontend Subnet: Subnet where the dedicated load balancer is located. The system allocates an IP address from this subnet to the load balancer for external services. After a load balancer is created, you can unbind the IP address from it and assign an IP address from a new frontend subnet to the load balancer. Backend Subnet: The load balancer uses IP addresses in the backend subnet to establish connections with backend servers.	Private IPv4 network HSS-outside-anp-VPC (VPC created in Step 1: Create a VPC) HSS-outside-subnet (VPC subnet created in Step 1: Create a VPC) Subnet of the load balancer
   Elastic IPs	Specifies the public IP address that will be bound to the load balancer for receiving and forwarding requests over the Internet.	New EIP Dynamic BGP Dedicated

3. After setting the parameters, click Next.
4. On the ELB page, view the created ELB and record the public IPv4 address.
5. In the row of a load balancer, click Add now in the Listener (Frontend Protocol/Port) column.
6. Set the listener parameters as prompted.
   You are advised to configure some parameters by referring to Table 6 and configure other parameters based on site requirements.

   Table 6 Parameters for adding a listener

   Parameter		Description	Example Value
   Configure Listener	Name	Listener name.	HSS-outside-anp-Listener
   	Protocol	Protocol used by the client and listener to distribute traffic.	TCP
   	Frontend Port	Port used by the client and listener to distribute traffic.	8091
   	Access Control	Supports access control based on the whitelist and blacklist.	All IP addresses
   Configure Routing Policy	Backend Server Group	A group of backend servers with the same features. New Use existing	New
   	Name	Specifies the name of the backend server group.	HSS-outside-anp-server-group
   	Backend Protocol	Specifies the protocol that backend servers in the backend server group use to receive requests from the listeners. The protocol varies depending on the forwarding mode:	TCP
   	Load Balancing Algorithm	Algorithm used by the load balancer. Weighted round robin: Requests are routed to different servers based on their weights. Backend servers with higher weights receive proportionately more requests, whereas equal-weighted servers receive the same number of requests. Weighted least connections: In addition to the number of connections, each server is assigned a weight based on its capacity. Requests are routed to the server with the lowest connections-to-weight ratio. Source IP hash: Allows requests from different clients to be routed based on source IP addresses and ensures that requests from the same client are forwarded to the same server.	Weighted Round Robin
   Add Backend Server	Backend Servers	When you use ELB to route requests, ensure that at least one backend server is running properly and can receive requests routed by the load balancer. Click Add Backend Servers.	HSS-outside-anp-ECS Set the service port to 8091. (Server created in Step 3: Create an ECS)

7. On the page for confirming the configuration, check parameter settings.
8. Click Submit complete the configuration.

Step 6: Modify a Security Group

1. Click in the upper left corner of the management console and choose Network > Virtual Private Cloud.
2. In the navigation tree on the left, choose Security Groups.
3. Locate the security group created in Step 2: Create a Security Group and click Manage Rules.
4. Delete the IPv6 full passing rule.
5. Modify the IPv4 full bypass rule.
   1. Change the value of Protocol Port from Basic/All to Basic/Custom TCP and set the port number to 8091.
   2. Click OK.

Step 7: Prepare the kubeconfig File

The kubeconfig file specifies the cluster permissions assigned to HSS. The kubeconfig file configured using method 1 contains the cluster administrator permissions, whereas the file generated using method 2 contains only the permissions required by HSS. If you want to minimize HSS permissions, prepare the file using method 2.

• Method 1: configuring the default kubeconfig file
  The default kubeconfig file is in the $HOME/.kube/config directory. Perform the following operations to create a dedicated namespace for HSS:]

  1. Log in to a cluster node.
  2. Create the hss.yaml file and copy the following content to the file:

     {"metadata":{"name":"hss"},"apiVersion":"v1","kind":"Namespace"}

  3. Run the following command to create a namespace:

     kubectl apply -f hss.yaml

• Method 2: generating a kubeconfig file dedicated to HSS
  1. Create a dedicated namespace and an account for HSS.
     1. Log in to a cluster node.
     2. Create the hss-account.yaml file and copy the following content to the file:

        {"metadata":{"name":"hss"},"apiVersion":"v1","kind":"Namespace"}{"metadata":{"name":"hss-user","namespace":"hss"},"apiVersion":"v1","kind":"ServiceAccount"}{"metadata":{"name":"hss-user-token","namespace":"hss","annotations":{"kubernetes.io/service-account.name":"hss-user"}},"apiVersion":"v1","kind":"Secret","type":"kubernetes.io/service-account-token"}

     3. Run the following command to create a namespace and an account:

        kubectl apply -f hss-account.yaml

  2. Generate the kubeconfig file.
     1. Create the gen_kubeconfig.sh file and copy the following content to the file:

        #!/bin/bash
        
        KUBE_APISERVER=`kubectl config view  --output=jsonpath='{.clusters[].cluster.server}' | head -n1 `
        CLUSTER_NAME=`kubectl config view -o jsonpath='{.clusters[0].name}'`
        kubectl get secret hss-user-token -n hss -o yaml |grep ca.crt: | awk '{print $2}' |base64 -d >hss_ca_crt
        
        kubectl config set-cluster ${CLUSTER_NAME} --server=${KUBE_APISERVER}  --certificate-authority=hss_ca_crt  --embed-certs=true --kubeconfig=hss_kubeconfig.yaml
        kubectl config set-credentials hss-user --token=$(kubectl describe secret hss-user-token -n hss | awk '/token:/{print $2}') --kubeconfig=hss_kubeconfig.yaml
        kubectl config set-context hss-user@kubernetes --cluster=${CLUSTER_NAME} --user=hss-user --kubeconfig=hss_kubeconfig.yaml
        kubectl config use-context hss-user@kubernetes --kubeconfig=hss_kubeconfig.yaml

     2. Run the following command to generate the kubeconfig file named hss_kubeconfig.yaml:

        bash gen_kubeconfig.sh

Step 8: Install the Agent for a Third-Party Public Network Cluster

The image repositories used by a cluster are classified into public image repositories and private image repositories.

• Public network image repository: An image repository that can be accessed as long as it can connect to the Internet. It is usually provided by a third party and paid by enterprises.
• Private image repository: an image repository deployed and maintained by an enterprise. Only authorized users can access the image repository.

Install the agent for the cluster based on the image repository type.