Scenario

A node is a virtual or physical machine that provides computing resources. Sufficient nodes must be available in your project to ensure that operations, such as creating workloads, can be performed.

Prerequisites

• At least one cluster is available. For details on how to create a cluster, see Buying a CCE Cluster.
• A key pair has been created. The key pair will be used for identity authentication upon remote node login.

  If you use a password to log in to a node, skip this step. For details, see Creating a Key Pair.

Notes and Constraints

• During the node creation, software packages are downloaded from OBS using the domain name. You need to use a private DNS server to resolve the OBS domain name, and configure the subnet where the node resides with a private DNS server address. When you create a subnet, the private DNS server is used by default. If you change the subnet DNS, ensure that the DNS server in use can resolve the OBS domain name.
• Only KVM nodes can be created. Non-KVM nodes cannot be used after being created.
• Once a node is purchased, its AZ cannot be changed.
• Nodes purchased in the pay-per-use billing mode will be deleted after you delete them on the Resource Management > Nodes page on the CCE console. Yearly/monthly-billed nodes in a cluster cannot be deleted on the CCE console. You can choose Billing Center > My Orders in the upper right corner of the page to unsubscribe from the nodes.
• CCE supports GPUs through an add-on named gpu-beta. You need to install this add-on to use GPU-enabled nodes in your cluster.
• If the cluster network model is container tunnel network, the cluster cannot contain VM nodes, and the cluster version must be v1.13.10 or later. (Clusters using this network model can manage only one type of nodes.)
• If the network model is VPC network, the cluster version must be v1.11.7 or later. (Clusters using this network model can manage VM nodes and BMS nodes at the same time).

Procedure

1. Log in to the CCE console. Use either of the following methods to add a node:
   • In the navigation pane, choose Resource Management > Nodes. Select the cluster to which the node will belong and click Buy Node on the upper part of the node list page.
     Figure 1 Method 1 for opening the Buy Node page
     
   • In the navigation pane, choose Resource Management > Clusters. In the card view of the cluster to which you will add nodes, click Buy Node.
     Figure 2 Method 2 for opening the Buy Node page
     

2. Set Billing mode to Pay-per-use or Yearly/Monthly. This section uses the pay-per-use billing mode as an example.
3. Select a region and an AZ.
   • Current Region: geographic location of the nodes to be created.
   • AZ: Set this parameter based on the site requirements. An AZ is a physical region where resources use independent power supply and networks. AZs are physically isolated but interconnected through an internal network.
     You are advised to deploy worker nodes in different AZs after the cluster is created to make your workloads more reliable. When creating a cluster, you can deploy nodes only in one AZ.

     Figure 3 Worker nodes in different AZs
     

4. Configure node parameters.
   • Node Type
     • VM node: A VM node will be created in the cluster.
     • Bare-metal node: Nodes running on bare metal servers (called BMS nodes) cannot be created along with a cluster. You can add BMS nodes to a cluster only after the cluster is created.
       

       BMS nodes can be created only after the CCE cluster is created and meets the following conditions:

       • All IP addresses are not IPv6.
       • The cluster version is v1.11.7 or later (with VPC network used), or v1.13.10 or later (with tunnel network used).
       • BMS nodes in the cluster are billed on a yearly/monthly basis.

       For details on how to buy BMS nodes, see Buying a Node.

   • Node Name: Enter a node name. A node name contains 1 to 56 characters starting with a lowercase letter and not ending with a hyphen (-). Only lowercase letters, digits, and hyphens (-) are allowed.

     If you change the node name on the ECS console after the node is created, be sure to synchronize the new node name from ECS to CCE. For details, see Synchronizing Node Data.

   • Specifications: Select node specifications that best fit your business needs.
     • General-purpose: provides a balance of computing, memory, and network resources. It is a good choice for many applications, such as web servers, workload development, workload testing, and small-scale databases.
     • Memory-optimized: provides higher memory capacity than general-purpose nodes and is suitable for relational databases, NoSQL, and other workloads that are both memory-intensive and data-intensive.
     • General computing-basic: provides a balance of computing, memory, and network resources and uses the vCPU credit mechanism to ensure baseline computing performance. Nodes of this type are suitable for applications requiring burstable high performance, such as light-load web servers, enterprise R&D and testing environments, and low- and medium-performance databases.
     • GPU-accelerated: provides powerful floating-point computing and is suitable for real-time, highly concurrent massive computing. Graphical processing units (GPUs) of P series are suitable for deep learning, scientific computing, and CAE. GPUs of G series are suitable for 3D animation rendering and CAD. GPU-accelerated nodes can be created only in clusters of v1.11 or later.
     • High-performance computing: provides stable and ultra-high computing performance and is suitable for scientific computing and workloads that demand ultra-high computing power and throughput.
     • General computing-plus: provides stable performance and exclusive resources to enterprise-class workloads with high and stable computing performance.
     • Disk-intensive: supports local disk storage and provides high network performance. It is designed for workloads requiring high throughput and data switching, such as big data workloads.
     • Ultra-high I/O: delivers ultra-low SSD access latency and ultra-high IOPS performance. This type of specifications is ideal for high-performance relational databases, NoSQL databases (such as Cassandra and MongoDB), and Elasticsearch.
     • Ascend-accelerated: Ascend-accelerated nodes powered by HiSilicon Ascend 310 AI processors are applicable to scenarios such as image recognition, video processing, inference computing, and machine learning.
       

       • Currently, Ascend-accelerated nodes are available only in specific AZs.
       • Before using Ascend-accelerated nodes, you must install the huawei-npu add-on to ensure that workloads using Ascend 310 processors can run properly.
       • After the nodes are created, the Ascend processor driver is installed and the nodes are automatically restarted. During the restart, the nodes are temporarily unavailable. They will automatically recover after the restart.
     Figure 4 Selecting node specifications
     

     To ensure node stability, CCE automatically reserves some resources to run necessary system components. For details, see Formula for Calculating the Reserved Resources of a Node.

   • OS: Select an OS for the node to be created. In certain regions, only OSs are displayed and options Public image and Private image are unavailable.
     • Public image: Select an OS for the node.

       A public image is a standard, widely used image. It contains an OS and preinstalled public applications and is available to all users. For more information, see Overview.

     • Private image (OBT): A private image contains an OS or service data, preinstalled public applications, and the owner's private applications. It is available only to the user who created it. Private images are supported only for clusters of v1.15 or later.

       If no private image is available, create one by following the instructions provided in Using a Private Image to Build a Worker Node Image (OBT).

     • Shared image: A shared image is a private image shared by another user. For details, see Overview of Sharing Images.

     Reinstalling the OS or modifying OS configurations could make the node unavailable. Exercise caution when performing these operations. For details, see High-Risk Operations and Solutions.

   • System Disk: Set the system disk space of the worker node. The value ranges from 40GB to 1024 GB. The default value is 40GB.

     By default, system disks support High I/O (SAS) and Ultra-high I/O (SSD) EVS disks. For details, see EVS Disk Overview.

     Encryption: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function. This function is available only in certain regions.

     • Encryption is not selected by default.
     • After you select Encryption, you can select an existing key in the displayed Encryption Setting dialog box. If no key is available, click the link next to the drop-down box to create a key. After the key is created, click the refresh icon.
   • Data Disk: Set the data disk space of the worker node. The value ranges from 100 GB to 32,768 GB. The default value is 100 GB. The EVS disk types provided for the data disk are the same as those for the system disk.
     

     If the data disk is uninstalled or damaged, the Docker service becomes abnormal and the node becomes unavailable. You are advised not to delete the data disk.

     • LVM: If this option is selected, CCE data disks are managed by the Logical Volume Manager (LVM). On this condition, you can adjust the disk space allocation for different resources. This option is selected for the first disk by default and cannot be unselected. You can choose to enable or disable LVM for new data disks.
       • This option is selected by default, indicating that LVM management is enabled.
       • You can deselect the check box to disable LVM management.
         

         • Disk space of the data disks managed by LVM will be allocated according to the ratio you set.
         • When creating a node in a cluster of v1.13.10 or later, if LVM is not selected for a data disk, follow instructions in Adding a Second Data Disk to a Node in a CCE Cluster to fill in the pre-installation script and format the data disk. Otherwise, the data disk will still be managed by LVM.
         • When creating a node in a cluster earlier than v1.13.10, you must format the data disks that are not managed by LVM. Otherwise, either these data disks or the first data disk will be managed by LVM.
     • Encryption: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function.
       This function is supported only for clusters of v1.13.10 or later in certain regions, and is not displayed for clusters of v1.13.10 or earlier.

       • Encryption is not selected by default.
       • After you select Encryption, you can select an existing key in the displayed Encryption Setting dialog box. If no key is available, click the link next to the drop-down box to create a key. After the key is created, click the refresh icon.
     • Add Data Disk: Currently, a maximum of two data disks can be attached to a node. After the node is created, you can go to the ECS console to attach more data disks. This function is available only to clusters of certain versions.
     • Data disk space allocation: Click to specify the resource ratio for Kubernetes Space and User Space. Disk space of the data disks managed by LVM will be allocated according to the ratio you set. This function is available only to clusters of certain versions.
       • Kubernetes Space: You can specify the ratio of the data disk space for storing Docker and kubelet resources. Docker resources include the Docker working directory, Docker images, and image metadata. kubelet resources include pod configuration files, secrets, and emptyDirs.

         The Docker space cannot be less than 10%, and the space size cannot be less than 60 GB. The kubelet space cannot be less than 10%.

         The Docker space size is determined by your service requirements. For details, see Data Disk Space Allocation.

       • User Space: You can set the ratio of the disk space that is not allocated to Kubernetes resources and the path to which the user space is mounted.
         

         Note that the mount path cannot be /, /home/paas, /var/paas, /var/lib, /var/script, /var/log, /mnt/paas, or /opt/cloud, and cannot conflict with the system directories (such as bin, lib, home, root, boot, dev, etc, lost+found, mnt, proc, sbin, srv, tmp, var, media, opt, selinux, sys, and usr). Otherwise, the system or node installation will fail.

     If the cluster version is v1.13.10-r0 or later and the node specification is Disk-intensive or Ultra-high I/O, the following options are displayed for data disks:

     • EVS: Parameters are the same as those when the node type is not Disk-intensive or Ultra-high I/O. For details, see Data Disk above.
     • Local disk: Local disks may break down and do not ensure data reliability. It is recommended that you store service data in EVS disks, which are more reliable than local disks.
       Local disk parameters are as follows:

       • Disk Mode: If the node type is disk-intensive, the supported disk mode is HDD.If the node type is ultra-high I/O, the supported disk mode is SSD.
       • Read/Write Mode: When multiple local disks exist, you can set the read/write mode. The serial and sequential modes are supported. Sequential indicates that data is read and written in linear mode. When a disk is used up, the next disk is used. Serial indicates that data is read and written in striping mode, allowing multiple local disks to be read and written at the same time.
       • Kubernetes Space: You can specify the ratio of the data disk space for storing Docker and kubelet resources. Docker resources include the Docker working directory, Docker images, and image metadata. kubelet resources include pod configuration files, secrets, and emptyDirs.
       • User Space: You can set the ratio of the disk space that is not allocated to Kubernetes resources and the path to which the user space is mounted.

     

     • The ratio of disk space allocated to the Kubernetes space and user space must be equal to 100% in total. You can click to refresh the data after you have modified the ratio.
     • By default, disks run in the direct-lvm mode. If data disks are removed, the loop-lvm mode will be used and this will impair system stability.

     Figure 5 Setting a local disk
     
   • VPC: A VPC where the current cluster is located. This parameter cannot be changed and is displayed only for clusters of v1.13.10-r0 or later.
   • Subnet: A subnet improves network security by providing exclusive network resources that are isolated from other networks. You can select any subnet in the cluster VPC. Cluster nodes can belong to different subnets.

     During the node creation, software packages are downloaded from OBS using the domain name. You need to use a private DNS server to resolve the OBS domain name, and configure the subnet where the node resides with a private DNS server address. When you create a subnet, the private DNS server is used by default. If you change the subnet DNS, ensure that the DNS server in use can resolve the OBS domain name.

     When a node is added to an existing cluster, if an extended CIDR block is added to the VPC corresponding to the subnet and the subnet is an extended CIDR block, you need to add the following three security group rules to the master node security group (the group name is in the format of Cluster name-cce-control-Random number). These rules ensure that the nodes added to the cluster are available. (This step is not required if an extended CIDR block has been added to the VPC during cluster creation.)

     

5. EIP: an independent public IP address. If the nodes to be created require public network access, select Automatically assign or Use existing. This parameter is not displayed when IPv6 is enabled for the cluster.
   An EIP bound to the node allows public network access. EIP bandwidth can be modified at any time. An ECS without a bound EIP cannot access the Internet or be accessed by public networks. For details, see EIP Overview.

   • Do not use: A node without an EIP cannot be accessed from public networks. It can be used only as a cloud server for deploying services or clusters on a private network.
   • Automatically assign: An EIP with specified configurations is automatically assigned to each node. If the number of EIPs is less than the number of nodes, the EIPs are randomly bound to the nodes.

     Configure the EIP specifications, billing factor, bandwidth type, and bandwidth size as required. When creating an ECS, ensure that the elastic IP address quota is sufficient.

   • Use existing: Existing EIPs are assigned to the nodes to be created.

   

   By default, VPC's SNAT feature is disabled for CCE. If SNAT is enabled, you do not need to use EIPs to access public networks. For details about SNAT, see Custom Policies.

6. Shared Bandwidth: Select Do not use or Use existing. This parameter is displayed only when IPv6 is enabled for the cluster.

   An EIP bound to the node allows public network access. EIP bandwidth can be modified at any time. An ECS without a bound EIP cannot access the Internet or be accessed by public networks.

7. Login Mode: You can use a password or key pair.
   • Password: The default username is root. Enter the password for logging in to the node and confirm the password.

     Be sure to remember the password as you will need it when you log in to the node.

   • Key pair: Select the key pair used to log in to the node. You can select a shared key.

     A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click Create a key pair. For details on how to create a key pair, see Creating a Key Pair.

     

     When creating a node using a key pair, IAM users can select only the key pairs created by their own, regardless of whether these users are in the same group. For example, user B cannot use the key pair created by user A to create a node, and the key pair is not displayed in the drop-down list on the CCE console.

     Figure 6 Key pair
     

8. Advanced ECS Settings (optional): Click to show advanced ECS settings.
   • ECS Group: An ECS group logically groups ECSs. The ECSs in the same ECS group comply with the same policy associated with the ECS group.
     • Anti-affinity: ECSs in an ECS group are deployed on different physical hosts to improve service reliability.
     • Fault domain: ECSs in an ECS group are deployed in multiple failure domains so that a failure in one failure domain will not affect the ECSs in other failure domains, thereby improving service reliability. This option is displayed only when the environment supports failure domains. This option is not supported if a worker node is deployed in a random AZ.

     Select an existing ECS group, or click Create ECS Group to create one. After the ECS group is created, click the refresh button.

   • Resource Tags: By adding tags to resources, you can classify resources.

     You can create predefined tags in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use predefined tags to improve tag creation and migration efficiency. For details, see Creating Predefined Tags.

     CCE will automatically create the "CCE-Dynamic-Provisioning-Node=node id" tag. A maximum of 5 tags can be added.

   • Agency: An agency is created by a tenant administrator on the IAM console. By creating an agency, you can share your cloud server resources with another account, or entrust a more professional person or team to manage your resources. For details on how to create an agency, see Cloud Service Delegation. To authorize an ECS or BMS to call cloud services, select Cloud service as the agency type, click Select, and then select ECS BMS.
   • Pre-installation Script: Enter a maximum of 1,000 characters.

     The script will be executed before Kubernetes software is installed. Note that if the script is incorrect, Kubernetes software may fail to be installed. The script is usually used to format data disks.

   • Post-installation Script: Enter a maximum of 1,000 characters.

     The script will be executed after Kubernetes software is installed and will not affect the installation. The script is usually used to modify Docker parameters.

   • Subnet IP Address: Select Automatically assign IP address (recommended) or Manually assigning IP addresses.

9. Advanced Kubernetes Settings: (Optional) Click to show advanced cluster settings.
   • Max Pods: maximum number of pods that can be created on a node, including the system's default pods. If the cluster uses the VPC network model, the maximum value is determined by the number of IP addresses that can be allocated to containers on each node.

     This limit prevents the node from being overloaded by managing too many pods. For details, see Maximum Number of Pods That Can Be Created on a Node.

   • Maximum Data Space per Container: maximum data space that can be used by a container. The value ranges from 10 GB to 500 GB. If the value of this field is larger than the data disk space allocated to Docker resources, the latter will override the value specified here. Typically, 90% of the data disk space is allocated to Docker resources. This parameter is displayed only for clusters of v1.13.10-r0 and later.

10. Nodes: The value cannot exceed the management scale you select when configuring cluster parameters. Set this parameter based on service requirements and the remaining quota displayed on the page. Click to view the factors that affect the number of nodes to be added (depending on the factor with the minimum value). To apply for more quotas, click Increase quota.
11. Validity Duration: Set this parameter if you select the yearly/monthly billing mode.
12. Click Next: Confirm. After confirming that the configuration is correct, click Submit.

    If the billing mode is Yearly/Monthly, click Pay Now after confirming the configuration, and pay as prompted.

    The node list page is displayed. If the node status is Available, the node is added successfully. It takes about 6 to 10 minutes to create a node.

    

    • If the console indicates insufficient EIP quota during node creation, increase the quota by following the instructions provided in How Do I Troubleshoot Insufficient EIPs When a Node Is Added?
    • A cloud server is automatically created during node creation. If the cloud server fails to be created, a rollback process starts and is charged according to the pricing principles of cloud servers. If there is a rollback fee, go to Billing Center to unsubscribe from the cloud server.
    • Do not delete the security groups and related rules automatically configured during cluster creation. Otherwise, the cluster will exhibit unexpected behavior.

13. Click Back to Node List. The node has been created successfully if it changes to the Available state.
    Figure 7 Node list
    
    

    The allocatable resources are calculated based on the resource request value (Request), which indicates the upper limit of resources that can be requested by pods on this node, but does not indicate the actual available resources of the node.

    The calculation formula is as follows:

    • Allocatable CPUs = Total CPUs – Requested CPUs of all pods – Reserved CPUs for other resources
    • Allocatable memory = Total memory – Requested memory of all pods – Reserved memory for other resources