Cluster Management

CS can work with a shared cluster or with exclusive clusters. Exclusive clusters are physically isolated from shared clusters and exclusive clusters created by other tenants. Exclusive clusters are not affected by jobs running on other clusters. User-defined jobs can only run on exclusive clusters. To run a user-defined job, you must create an exclusive cluster.

This section describes the following:

Cluster List

All clusters are listed in the cluster list on the Cluster Management page. Table 1 describes parameters in the cluster list.

Table 1 Cluster list parameters

Parameter

Description

ID

ID of a cluster, which is automatically allocated during cluster creation

Name

Name of a cluster, which can be up to 100 characters long and can only contain letters, digits, hyphens (-), and underscores (_)

Status

Cluster status. The possible statuses are:

  • Requesting resources
  • Requesting resources failed
  • Requesting resources succeeded, cluster to be started
  • Starting
  • Running
  • Stopping
  • Stopping failed
  • Stopped
  • Restarting
  • Deleting
  • Deleted
  • Deletion failed
  • Stopping due to arrears
  • Stopping due to arrears failed
  • Stopped due to arrears
  • Restoring (recharged cluster)
  • Restoration of frozen cluster failed

Description

Description of a cluster. It can be up to 512 characters long.

SPU Usage (Used SPUs/SPU Quota)

SPU usage of a cluster. For example, 2/12, where 2 indicates the SPUs used by the cluster and 12 indicates the maximum SPUs allowed in the cluster.

Created

Time when a cluster is created

Enterprise Project

Enterprise project that a cluster belongs to

Billing Mode

Billing mode of a cluster, either Yearly/Monthly or Pay-per-use

Operation

  • Click Job Management to perform operations on all of the jobs in a cluster.
  • The following operations are allowed on clusters that are billed on a pay-per-use basis:
    • Click More > Test Address Connectivity to test whether the connection between the current cluster and the specified address is normal. The address can be a domain name plus IP address or a specified port.
    • Click More > Modify Max. SPUs to change the SPU quota of the cluster.
    • Click More > Delete to delete the cluster that has been created.
    • Click More > Stop to stop the target cluster.
    • Click More > Restart to restart the target cluster.
  • The following operations are allowed on clusters that are billed on a yearly/monthly basis:
    • Click More > Renew to renew the subscription to the target cluster.
    • Click More > Unsubscribe to unsubscribe from the target cluster.

Creating a Cluster Billed on a Yearly/Monthly Basis

  1. In the navigation tree on the left pane of the CS management console, click Cluster Management to switch to the Cluster Management page.
  1. Click Create Cluster to switch to the Create Cluster page.
  2. Specify cluster parameters.

    Figure 1 Creating a cluster
    Table 2 Cluster parameters

    Parameter

    Description

    Billing Mode

    Select Yearly/Monthly.

    Region

    Click in the upper left corner of the management console to select the actual region for the cluster.

    Currently, regions CN South-Guangzhou and CN North-Beijing1 are supported.

    Name

    Name of a cluster, which can be up to 100 characters long and can only contain letters, digits, hyphens (-), and underscores (_)

    NOTE:

    The cluster name must be globally unique.

    Description

    Description of a cluster. It can be up to 512 characters long.

    Management Node Specs

    Each tenant cluster consists of three management nodes. By default, each management node contains two SPUs (2 vCPUs and 8 GB memory).

    Available options include 2 SPUs, 4 SPUs, 8 SPUs, 16 SPUs, and 32 SPUs.

    SPUs

    Total number of SPUs required by the management nodes and jobs of a cluster. The maximum value of this parameter is 400 and its minimum value is calculated as follows: SPUs configured for a management node x 3 + 2.

    Required Duration

    Select the amount of time you want to use CS for.

    NOTE:
    • After you specify Required Duration and SPUs, the system automatically calculates the fees you need to pay.
    • Under Configuration Price, you can click Pricing details to view the detailed price.
    • If you want to renew the cluster after the required duration expires, select Auto-renew. For a monthly subscription, the subscription is automatically renewed every month. For a yearly subscription, the subscription is automatically renewed every year.

    Advanced Configuration

    You can configure and adjust the VPC and subnet to which the cluster belongs based on the network plan.

    Currently, the following network segments are supported: 10.0.0.0/8–24, 172.16.0.0/12–24, and 192.168.0.0/16–24.

  3. For Enterprise Project, select the enterprise project that you created on the Enterprise Management console.

    For details about how to create an enterprise project on the Enterprise Management console, see Creating an Enterprise Project in the Enterprise Management User Guide.

    The system also has a built-in enterprise project, default. If you do not select an enterprise project for the cluster, the default project is used instead.

    During cluster creation, if the cluster is successfully bound to an enterprise project, the cluster has been created. If the binding fails, the system sends an alarm and the cluster fails to be created.

    When you delete a cluster, the association between the cluster and its enterprise project is automatically deleted as well.

  4. (Optional) Add tags for the cluster. In this step, configure the parameters in the following table as required. The tags are optional. If you do not need tags, skip this step.

    Table 3 Tag parameters

    Parameter

    Description

    Tag key

    You can perform the following operations:

    • Click the text box and select a predefined tag key from the drop-down list.
      NOTE:

      To add a predefined tag, you need to create one on TMS and select it from the Tag key drop-down list. You can click View Predefined Tag to enter the Predefined Tag page of TMS. Then, click Create Tag to create a predefined tag. For details, see section Creating Predefined Tags in the Tag Management Service User Guide.

    • Enter a tag key in the text box.
      NOTE:

      A tag key contains a maximum of 36 characters. The first and last characters cannot be spaces. The following characters are not allowed: =*,<>\|/

    Tag value

    You can perform the following operations:

    • Click the text box and select a predefined tag value from the drop-down list.
    • Enter a tag value in the text box.
      NOTE:

      A tag value contains a maximum of 43 characters. The first and last characters cannot be spaces. The following characters are not allowed: =*,<>\|/

    • A maximum of 10 tags can be added.
    • Only one tag value can be added to a tag key.
    • The key name must be unique in the same resource.

  5. Click OK. On the displayed page, select any of the following payment methods: Balance, Online Payment, and Request Online Contract and Pay.
  6. Click Pay or Create contract.

    Switch to the Cluster Management page, where you can see that Status of the created cluster is Requesting resources.

    It takes about 1 to 3 minutes to create a cluster. If the value of Status changes to Running, the cluster has been created.

Creating a Cluster Billed on a Per-per-Use Basis

  1. In the navigation tree on the left pane of the CS management console, click Cluster Management to switch to the Cluster Management page.
  1. Click Create Cluster to switch to the Create Cluster page.
  2. Specify cluster parameters.

    Figure 2 Creating a cluster
    Table 4 Cluster parameters

    Parameter

    Description

    Billing Mode

    Select Pay-per-use.

    Region

    Click in the upper left corner of the management console to select the actual region for the cluster.

    Currently, regions CN South-Guangzhou and CN North-Beijing1 are supported.

    Name

    Name of a cluster, which contains 1 to 100 characters and can only contain letters, digits, hyphens (-), and underscores (_)

    NOTE:

    The cluster name must be globally unique.

    Description

    Description of a cluster. It can be up to 512 characters long.

    Management Node Specs

    Each tenant cluster consists of three management nodes. By default, each management node contains two SPUs (2 vCPUs and 8 GB memory). The SPU specifications of a management node cannot be modified after being created.

    Available options include 2 SPUs, 4 SPUs, 8 SPUs, 16 SPUs, and 32 SPUs.

    Max. SPUs in a Cluster

    Maximum number of SPUs in a cluster (excluding the SPUs consumed by the management nodes). The value ranges from 1 to 800. The default value is 100.

    Advanced Settings

    You can configure and adjust the VPC and subnet that the cluster belongs to based on the network plan.

    Currently, the following network segments are supported: 10.0.0.0/8–24, 172.16.0.0/12–24, and 192.168.0.0/16–24.

  3. For Enterprise Project, select the enterprise project that you created on the Enterprise Management console.

    For details about how to create an enterprise project on the Enterprise Management console, see Creating an Enterprise Project in the Enterprise Management User Guide.

    The system also has a built-in enterprise project, default. If you do not select an enterprise project for the cluster, the default project is used instead.

    During cluster creation, if the cluster is successfully bound to an enterprise project, the cluster has been created. If the binding fails, the system sends an alarm and the cluster fails to be created.

    When you delete a cluster, the association between the cluster and its enterprise project is automatically deleted as well.

  4. (Optional) Add tags for the cluster. In this step, configure the parameters in the following table as required. The tags are optional. If you do not need tags, skip this step.

    Table 5 Tag parameters

    Parameter

    Description

    Tag key

    You can perform the following operations:

    • Click the text box and select a predefined tag key from the drop-down list.
      NOTE:

      To add a predefined tag, you need to create one on TMS and select it from the Tag key drop-down list. You can click View Predefined Tag to enter the Predefined Tag page of TMS. Then, click Create Tag to create a predefined tag. For details, see section Creating Predefined Tags in the Tag Management Service User Guide.

    • Enter a tag key in the text box.
      NOTE:

      A tag key contains a maximum of 36 characters. The first and last characters cannot be spaces. The following characters are not allowed: =*,<>\|/

    Tag value

    You can perform the following operations:

    • Click the text box and select a predefined tag value from the drop-down list.
    • Enter a tag value in the text box.
      NOTE:

      A tag value contains a maximum of 43 characters. The first and last characters cannot be spaces. The following characters are not allowed: =*,<>\|/

    • A maximum of 10 tags can be added.
    • Only one tag value can be added to a tag key.
    • The key name must be unique in the same resource.

  5. Click OK. The system automatically switches to the Cluster Management page, on which you can see that Status of the created cluster is Requesting resources.

    It takes about 1 to 3 minutes to create a cluster. If the value of Status changes to Running, the cluster has been created.

Viewing Cluster Details

You can view the detailed information about created clusters.

  1. In the navigation tree on the left pane of the CS management console, click Cluster Management to switch to the Cluster Management page.
  2. In the row where the cluster you want to view is located, click the cluster name in the Name column to switch to the Cluster Details page.

    On the page showing the cluster details, you can view detailed information about the current cluster.

    • Click Test Address Connectivity. In the dialog box that is displayed, enter the address to be tested and click OK to test whether the connection between the current cluster and the specified address is normal. The address can be a domain name plus IP address or a specified port.
    • Click Modify Max. SPUs to modify the SPU quota of the cluster that is billed on a pay-per-use basis.
    • Click More for a cluster billed on pay-per-use basis to perform related operations, such as managing jobs in the cluster or stopping or restarting the cluster.
    • For a cluster billed on yearly/monthly basis, click More to perform related operations, such as managing jobs in the cluster or renewing or unsubscribing from the cluster.

  3. Click VPC Peering Connections to display information about the VPC peering connections of the current cluster.

    For details about the VPC peering connection, see VPC Peering Connection.

  4. Click IP-Domain Mapping to display information about IP-domain mapping of the current cluster.

    For details about how to create an IP-domain mapping, see Creating an IP-Domain Mapping.

  5. Click File Uploading to display the files that have been uploaded to the cluster.

    For details about how to upload files to a cluster, see Uploading Files.

  6. Click Audit Logs to display operation logs of the current cluster.

    For details about cluster audit logs, see Viewing Cluster Audit Logs.

  7. Click Tags to display the tag information about the current cluster.

    For details about the cluster tags, see Managing Cluster Tags.

Creating an IP-Domain Mapping

After creating a cluster, you can create an IP-domain mapping for the cluster to connect to other services.

  1. In the navigation tree on the left pane of the CS management console, click Cluster Management to switch to the Cluster Management page.
  2. In the row where the target cluster is located, click the cluster name in the Name column to switch to the Cluster Details page.
  3. Click IP-Domain Mapping to display information about IP-domain mapping of the current cluster.
  4. Click Create IP-Domain Mapping. In the displayed dialog box, specify Domain Name and IP Address and click OK.

    Figure 3 Creating an IP-domain mapping

    After an IP-domain mapping is created, the current CS cluster can interconnect with the mapped IP address.

    • The domain name can contain only letters, digits, hyphens (-), and dots (.), and must start or end with a maximum of letters or digits. It contains a maximum of 67 characters.
    • To edit an IP-domain mapping in the mapping list, locate the row where the IP-domain mapping is located and click Edit in the Operation column.
    • To delete an IP-domain mapping in the mapping list, locate the row where the IP-domain mapping is located and click Delete in the Operation column.

  5. (Optional) Click Add hosts File to add IP-domain mapping for the cluster.

    Ensure that each line in the hosts file is in the "ip hostname" format. If an IP address corresponds to multiple domain names, the line can be in the "ip hostname1 hostname2" format.

Uploading Files

You can use OBS to upload custom files (such as configuration files required by CS jobs) to the CS cluster.

Upload the desired files to the OBS bucket first. For details about how to upload files to the OBS bucket, see Uploading a File in the Object Storage Service Console Operation Guide.

To upload a file to a cluster, perform the following operations:

  1. In the navigation tree on the left pane of the CS management console, click Cluster Management to switch to the Cluster Management page.
  2. In the row where the cluster where you want to upload a file is located, click the cluster name in the Name column to switch to the Cluster Details page.
  3. Click File Uploading to display the files that have been uploaded to the cluster.
  4. Click Upload File. In the displayed dialog box, select the target file and click OK.

    The uploaded file is in the /opt/cs/user_files directory of the ECS that the cluster belongs to.

Modifying a Cluster

You can modify the cluster name, description, and SPU quota of a cluster in Running status.

  1. In the navigation tree on the left pane of the CS management console, click Cluster Management to switch to the Cluster Management page.
  2. In the row where the cluster you want to modify is located, click the cluster name in the Name column to switch to the Cluster Details page.
  3. Click the editing icon next to Name to change the cluster name.
  4. Click the editing icon next to Description to modify the description of the cluster.
  5. Click Modify Max. SPUs for a cluster billed on pay-per-use basis to modify the SPU quota of the cluster.

    • If the used SPUs reach the cluster's SPU quota, increase the SPU quota. Otherwise, you cannot create jobs in the cluster.
    • The maximum value of Max. SPUs in a Cluster is 800. If you want to increase Max. SPUs in a Cluster to a value greater than 800, click Service Tickets > Create Service Ticket in the upper right corner of the management console to request SPU quota increase.

Job Management

You can manage individual jobs within a cluster. On the Job Management page in Cluster Management, you can only view, start, stop, and delete jobs in a cluster, visualize streaming data in jobs, and modify job names and descriptions. For details, see Performing Operations on a Job.

  1. In the navigation tree on the left pane of the CS management console, click Cluster Management to switch to the Cluster Management page.
  2. Locate the row where the target cluster is located, click Manage Job in the Operation column to switch to the Job Management page of the cluster.

    Only the jobs for which the cluster is selected during job creation are displayed.

Stopping a Cluster

To stop a cluster, perform the following operations:

After a cluster is stopped, all jobs in the cluster are stopped. Exercise caution when performing this operation.

You can stop only the clusters that are billed on a pay-per-use basis.

  1. In the navigation tree on the left pane of the CS management console, click Cluster Management to switch to the Cluster Management page.
  2. Locate the row where the cluster you want to delete is located, and click More > Stop in the Operation column to switch to the Stop Cluster page.
  3. Click OK.

Restarting a Cluster

To restart a stopped cluster, perform the following operations:

You can restart only the clusters that are billed on a pay-per-use basis.

  1. In the navigation tree on the left pane of the CS management console, click Cluster Management to switch to the Cluster Management page.
  2. Locate the row where the cluster you want to restart is located, and click More > Restart in the Operation column.

    It takes 1 to 3 minutes to restart the cluster.

Deleting a Cluster

If you do not need to use a cluster, perform the following operations to delete it.

A deleted cluster cannot be restored and all jobs in the deleted cluster will be stopped. Exercise caution when performing this operation.

You can delete only the clusters that are billed on a pay-per-use basis. If you no longer use a cluster billed on a yearly/monthly basis, you can unsubscribe from it. For details, see Unsubscribing from a Cluster.

  1. In the navigation tree on the left pane of the CS management console, click Cluster Management to switch to the Cluster Management page.
  2. Locate the row where the cluster you want to delete is located, and click Delete in the Operation column to switch to the Delete Cluster dialog box.
  3. Click OK.

Renewing a Cluster

You can only renew clusters that are billed on a yearly/monthly basis.

  1. In the navigation tree on the left pane of the CS management console, click Cluster Management to switch to the Cluster Management page.
  2. In the cluster list, locate the row where the target cluster is located and click More > Renew in the Operation column.
  3. In the displayed dialog box, click OK to switch to the Renew page.
  4. Select the renewal duration and click Pay.
  5. On the displayed page, select any of the following payment methods: Balance, Online Payment, and Request Online Contract and Pay.
  6. Click Pay or Create contract.

    On the menu bar of the management console, you can choose Billing > My Orders to view the renewal order details.

Unsubscribing from a Cluster

You can unsubscribe from only the clusters that are billed on a yearly/monthly basis. For details about unsubscription rules and handling fees, see Unsubscription Rules.

After you unsubscribe from a cluster, resources and data will be deleted and cannot be retrieved. Therefore, ensure that the data has been backed up before the unsubscription.

  1. In the navigation tree on the left pane of the CS management console, click Cluster Management to switch to the Cluster Management page.
  2. In the cluster list, locate the row where the target cluster resides and click More > Unsubscribe in the Operation column.
  3. Click OK to switch to the Unsubscribe page.
  4. Select the unsubscription reason and click Confirm.
  5. In the displayed dialog box, click OK.

    Switch to the Cluster Management page. Status of the cluster that is successfully unsubscribed from is Stopped due to arrears.

    On the menu bar of the management console, you can choose Billing > My Orders to view the unsubscription order details.