Help Center> Cloud Container Engine> API Reference> Appendix> Specifying Add-ons to Be Installed During Cluster Creation
Updated on 2024-05-30 GMT+08:00
View PDF

Specifying Add-ons to Be Installed During Cluster Creation

When creating a cluster, you can add a key-value pair to annotations of the metadata field in the request body to install an add-on in the cluster. The key is cluster.install.addons/install, and the value is a JSON array of AddonTemplate.

Table 1 Value data structure

Parameter

Mandatory

Type

Description

Value

Yes

JSON array of AddonTemplate string

Add-on to be installed in the cluster. If this parameter is not specified, the coredns and storage-driver add-ons are installed by default for clusters of Kubernetes 1.13 and earlier versions, and the coredns, everest, and npd add-ons are installed by default for clusters of Kubernetes 1.15 and later versions. For details, see Table 2.
Table 2 Data structure of the AddonTemplate field

Parameter

Mandatory

Type

Description

addonTemplateName

Yes

String

Add-on name.

The value can be coredns, autoscaler, gpu-beta, or storage-driver.

version

No

String

Add-on version.

To view the add-on version information, log in to the CCE console, click Add-ons in the navigation pane. Click the target add-on name. On the Version History tab page, you can view the versions of the add-on. If this parameter is left blank, the latest version is used by default.

values

No

Json Map

Parameters to be set for installing an add-on.

  • coredns: For details, see Table 3.
  • autoscaler: For details, see Table 7.
  • gpu-beta: For details, see Table 9.
NOTE:

You do not need to set this parameter when installing the storage-driver add-on.
Table 3 Special key-value pairs of the coredns values parameters

Parameter

Mandatory

Type

Description

flavor

Yes

flavor object

Add-on specifications.

custom

Yes

custom object

Custom coredns parameters.
Table 4 Data structure of the flavor field

Parameter

Mandatory

Type

Description

description

Yes

String

Specification description.

name

Yes

String

Flavor name.

replicas

Yes

Integer

Number of pods.

resources

Yes

resource object

Add-on resources.
Table 5 Data structure of the resource field

Parameter

Mandatory

Type

Description

name

Yes

String

Resource name.

limitsCpu

Yes

String

CPU quota limit.

limitsMem

Yes

String

Memory quota limit.

requestsCpu

Yes

String

Requested CPU.

requestsMem

Yes

String

Requested memory.
Table 6 Data structure of the custom field

Parameter

Mandatory

Type

Description

stub_domains

No

Json Map

JSON map that uses the DNS suffix key (such as acme.local) and the value composed of the JSON array of the DNS IP address.

upstream_nameservers

No

Json Array

The value of this parameter replaces the name server value obtained from /etc/resolv.conf of the node by default. Restriction: A maximum of three upstream name servers can be specified.

tenant_id

Yes

String

Project ID.
Table 7 Special key-value pairs of the autoscaler values parameters

Parameter

Mandatory

Type

Description

flavor

Yes

flavor Object

Add-on specifications.

custom

Yes

custom object

Custom autoscaler parameters.
Table 8 Data structure of the custom field

Parameter

Mandatory

Type

Description

coresTotal

Yes

Integer

Maximum and minimum number of cores of a cluster. The format is <min>:<max>. The autoscaler add-on performs cluster scaling within this range.

maxEmptyBulkDeleteFlag

Yes

Integer

Maximum number of empty nodes that can be deleted at a time.

maxNodesTotal

Yes

Integer

Maximum number of nodes in all node groups. The autoscaler add-on performs cluster scaling within this range.

memoryTotal

Yes

Integer

Maximum and minimum size of memory of a cluster. The format is <min>:<max>. The autoscaler add-on performs cluster scaling within this range.

scaleDownDelayAfterAdd

Yes

Integer

Interval for performing scale-down evaluation after a scale-up.

scaleDownDelayAfterDelete

Yes

Integer

Interval for performing scale-down evaluation after nodes are deleted. The default value is scanInterval.

scaleDownDelayAfterFailure

Yes

Integer

Interval for performing scale-down evaluation after a scale-down failure.

scaleDownEnabled

Yes

Boolean

Whether to enable cluster scaling down.

scaleDownUnneededTime

Yes

Integer

Time waited to start a scale-down after a node has reached the scale-down threshold.

scaleDownUtilizationThreshold

Yes

Float

Threshold percentage of the total CPU or memory occupied by all pods running on the node to the allocatable resources on the node. When the actual percentage is lower than this threshold, node scale-down is triggered.

scaleUpCpuUtilizationThreshold

Yes

Float

CPU usage threshold for triggering scale-up.

scaleUpMemUtilizationThreshold

Yes

Float

Memory usage threshold for triggering scale-up.

scaleUpUnscheduledPodEnabled

Yes

Boolean

Scale-up is triggered when there are unscheduled pods.

scaleUpUtilizationEnabled

Yes

Boolean

Scale-up is triggered when the resource usage reaches the threshold.

tenant_id

Yes

String

Project ID.

unremovableNodeRecheckTimeout

Yes

Integer

Interval for checking whether a node can be removed.
Table 9 Special Key-Value in values of AddonTemplate (GPU-beta)

Parameter

Mandatory

Type

Description

custom

Yes

custom object

Customer parameters of gpu-beta.
Table 10 Data structure of the custom field

Parameter

Mandatory

Type

Description

is_driver_from_nvidia

Yes

Boolean

Whether to use the NVIDIA driver.

Set this parameter to true.

nvidia_driver_download_url

Yes

String

Driver downloading URL.
Table 11 Data structure of the spec field

Parameter

Mandatory

Type

Description

type

Yes

String

Cluster type. Possible values:

  • VirtualMachine: The cluster is a hybrid cluster.

    A hybrid cluster manages a group of node resources based on Kubernetes. It can manage VMs, bare-metal machines, or a combination of both. Kubernetes automatically schedules containers onto available nodes. Before creating a containerized workload, you must have an available cluster.

  • ARM64: The cluster is a Kunpeng cluster.

    Containers in CCE's Kunpeng clusters can run on Kunpeng servers that use ARM architecture and Kunpeng processors. Kunpeng-accelerated cloud servers are easy to deploy and provide comparable scaling and scheduling performance as x86-based cloud servers.

flavor

Yes

String

Cluster flavor, which cannot be changed after the cluster is created.

  • cce.s1.small: a small-scale CCE cluster with one master node and a maximum of 50 worker nodes
  • cce.s1.medium: a medium-scale CCE cluster with one master node and a maximum of 200 worker nodes
  • cce.s2.small: a small-scale, HA CCE cluster with three master nodes and a maximum of 50 worker nodes
  • cce.s2.medium: a medium-scale, HA CCE cluster with three master nodes and a maximum of 200 worker nodes
  • cce.s2.large: a large-scale, HA CCE cluster with three master nodes and a maximum of 1,000 worker nodes
  • cce.s2.xlarge: an ultra-large-scale, HA CCE cluster with three master nodes and a maximum of 2,000 worker nodes
NOTE:
  • s1: indicates a cluster with one master node. If the master node is faulty, the cluster is unavailable, but running workloads are not affected.
  • s2: indicates a cluster with three master nodes. This is an HA cluster. If one of the master nodes is faulty, the cluster is still available.
  • dec: dedicated hybrid cluster specifications. For example, cce.dec.s1.small is a small-scale, single-master, dedicated hybrid cluster (≤ 50 nodes).
  • For example, ≤ 50 nodes indicates that the maximum number of nodes that can be managed by the cluster is 50.
  • A single-master cluster has only one master node. If the master node is down, the cluster will become unavailable and stop serving new workloads. However, existing workloads in the cluster are not affected.
  • An HA cluster has multiple master nodes. Faults in a single master node will not take the cluster down. For details on how to distinguish high-availability clusters from regular clusters, see How Do I Check Whether a Cluster Is in Multi-Master Mode?

version

No

String

Cluster's baseline Kubernetes version. The latest version is recommended.

  • You can create clusters of two latest versions on the CCE console. To learn which cluster versions are available, choose Clusters in the navigation pane, click Buy Cluster, and view the supported cluster versions.
  • You can call APIs to create clusters of other versions. However, these clusters will be gradually brought offline. For details about the offline policy, see the official CCE announcement.
NOTE:

If this parameter is left blank, the cluster of the latest version is created by default.

description

No

String

Cluster description, for example, which purpose the cluster is intended to serve. By default, this parameter is left unspecified. To modify cluster description after the cluster is created, call the API used to update information about a specified cluster or go to the cluster details page on the CCE console.

ipv6enable

No

Boolean

Whether the cluster supports IPv6. Clusters of version 1.15 and later support IPv6.

hostNetwork

Yes

HostNetwork object

Node networking parameters, including VPC and subnet ID. hostNetwork is mandatory because nodes in a cluster communicate with each other by using a VPC.

containerNetwork

Yes

ContainerNetwork object

Container networking parameters, including the container network model and container CIDR block.

authentication

No

Authentication object

Configurations of the cluster authentication mode.

kubernetesSvcIpRange

No

String

Service CIDR block or the IP address range which the kubernetes clusterIp must fall within. This parameter is available only for clusters of v1.11.7 and later.

billingMode

No

Integer

Billing mode of a cluster. Currently, only pay-per-use clusters can be created.

Value 0 indicates pay-per-use. If this parameter is left unspecified, the default value 0 is used.

masters

No

MasterSpec object

Advanced configurations of the master node

extendParam

No

Map<String,String>

Extended fields in the format of key-value pairs.

You can configure multi-AZ clusters and dedicated hybrid clusters, and create clusters in specific enterprise projects.

Request Example

{
  "kind" : "Cluster",
  "apiVersion" : "v3",
  "metadata" : {
    "name" : "mycluster",
    "labels" : {
      "foo" : "bar"
    },
    "annotations" : {
      "cluster.install.addons/install" : "[{\"addonTemplateName\":\"autoscaler\",\"version\":\"1.15.11\",\"values\":{\"flavor\":{\"description\":\"Has only one instance\",\"name\":\"Single\",\"replicas\":1,\"resources\":[{\"limitsCpu\":\"90m\",\"limitsMem\":\"200Mi\",\"name\":\"autoscaler\",\"requestsCpu\":\"50m\",\"requestsMem\":\"100Mi\"}]},\"custom\":{\"coresTotal\":32000,\"maxEmptyBulkDeleteFlag\":10,\"maxNodesTotal\":1000,\"memoryTotal\":128000,\"scaleDownDelayAfterAdd\":10,\"scaleDownDelayAfterDelete\":10,\"scaleDownDelayAfterFailure\":3,\"scaleDownEnabled\":false,\"scaleDownUnneededTime\":10,\"scaleDownUtilizationThreshold\":0.5,\"scaleUpCpuUtilizationThreshold\":1,\"scaleUpMemUtilizationThreshold\":1,\"scaleUpUnscheduledPodEnabled\":true,\"scaleUpUtilizationEnabled\":true,\"tenant_id\":\"22a8a02394794b908d256103a5b63c65\",\"unremovableNodeRecheckTimeout\":5}}},{\"addonTemplateName\":\"coredns\",\"version\":\"1.15.3\",\"values\":{\"flavor\":{\"description\":\"Has two instances\",\"name\":2500,\"replicas\":2,\"resources\":[{\"limitsCpu\":\"500m\",\"limitsMem\":\"512Mi\",\"name\":\"coredns\",\"requestsCpu\":\"500m\",\"requestsMem\":\"512Mi\"}]},\"custom\":{\"stub_domains\":{},\"upstream_nameservers\":[],\"tenant_id\":\"22a8a02394794b908d256103a5b63c65\"}}},{\"addonTemplateName\":\"gpu-beta\",\"version\":\"1.1.7\",\"values\":{\"custom\":{\"is_driver_from_nvidia\":true,\"nvidia_driver_download_url\":\"https://us.download.nvidia.com/tesla/396.37/NVIDIA-Linux-x86_64-396.37.run\"}}},{\"addonTemplateName\":\"storage-driver\",\"version\":\"\"}]"
    }
  },
  "spec" : {
    "type" : "VirtualMachine",
    "flavor" : "cce.s1.small",
    "version" : "v1.15.11",
    "description" : "this is a demo cluster",
    "hostNetwork" : {
      "vpc" : "1cb74d47-8e09-4d14-a065-75f4fc03e5eb",
      "subnet" : "d44c089c-fbdc-49b3-ae8f-7b42c64219cf"
    },
    "containerNetwork" : {
      "mode" : "overlay_l2",
      "cidr" : "172.16.0.0/16"
    },
    "extendParam" : {
      "foo" : "bar"
    }
  }
}
Parent Topic: Appendix