Help Center/ MapReduce Service/ API Reference/ API V1.1/ Cluster Management APIs/ Creating a Cluster and Executing a Job - CreateClusterAndSubmitJob
Updated on 2026-09-15 GMT+08:00
View PDF
Share

Creating a Cluster and Executing a Job - CreateClusterAndSubmitJob

Function

This API is used to create an MRS cluster and submit a job in the cluster. This API is incompatible with Sahara.

You are advised to use the V2 APIs for creating a cluster and creating a cluster and submitting a job.

A maximum of 10 clusters can be created concurrently at the same time.

Before using this API, you need to obtain the resource information as shown in Table 1.

Table 1 Obtaining resources

Resource

How to Obtain

VPC

See Querying VPCs and Creating a VPC in Virtual Private Cloud (VPC) API Reference.

Subnet

See Querying Subnets and Creating a Subnet in Virtual Private Cloud (VPC) API Reference.

Key Pair

See Querying SSH Key Pairs and Creating and Importing an SSH Key Pair in Elastic Cloud Server (ECS) API Reference.

Zone

See Endpoints for details about regions and AZs.

Version

Currently, MRS 1.9.2, 3.1.0, 3.1.2-LTS.3, 3.1.5, 3.2.0-LTS.1, 3.3.0-LTS, 3.6.0-LTS, and 3.5.0-LTS are supported.

Component
  • MRS 3.6.0-LTS supports the following components:
    • A custom cluster can contain the following components: Hadoop, Spark, HBase, Hive, Kafka, Flume, Flink, Oozie, ZooKeeper, HetuEngine, Ranger, Tez, JobGateway, ClickHouse, Guardian, MemArtsCC, Sqoop, Kudu, Impala, and MemArtsStore.
    • A Doris cluster can contain the following components:

      Doris

    • An analysis cluster can contain the following components: Hadoop, Spark, HBase, Hive, Loader, Flink, Oozie, ZooKeeper, HetuEngine, Ranger, Tez, JobGateway, Guardian, MemArtsCC, Sqoop, Kudu, Impala, and MemArtsStore.
    • A streaming cluster can contain the following components: Kafka, Flume, ZooKeeper, and Ranger.
    • A hybrid cluster can contain the following components: Hadoop, Spark, HBase, Hive, Flink, Oozie, ZooKeeper, HetuEngine, Ranger, Tez, Kafka, Flume, JobGateway, Guardian, MemArtsCC, Sqoop, Kudu, Impala, and MemArtsStore.
  • MRS 3.5.0-LTS supports the following components:
    • A custom cluster can contain the following components: Hadoop, Spark, HBase, Hive, Kafka, Flume, Flink, Oozie, ZooKeeper, HetuEngine, Ranger, Tez, ClickHouse, Guardian, JobGateway, MemArtsCC, Sqoop, Kudu, and Impala.
    • A Doris cluster can contain the following components:

      Doris

    • An analysis cluster contains the following components: Hadoop, Spark, HBase, Hive, Loader, Flink, Oozie, ZooKeeper, HetuEngine, Ranger, Tez, JobGateway, Guardian, MemArtsCC, Sqoop, Kudu, and Impala.
    • A streaming cluster contains the following components: Kafka, Flume, ZooKeeper, and Ranger.
    • A hybrid cluster contains the following components: Hadoop, Spark, HBase, Hive, Flink, Oozie, ZooKeeper, HetuEngine, Ranger, Tez, Kafka, Flume, JobGateway, Guardian, MemArtsCC, Sqoop, Kudu, and Impala.
  • MRS 3.3.0-LTS supports the following components:
    • A custom cluster can contain the following components: CDL, Hadoop, Spark, HBase, Hive, IoTDB, Loader, Kafka, Flume, Flink, Oozie, ZooKeeper, HetuEngine, Ranger, Tez, ClickHouse, Guardian, and JobGateway.
    • A Doris cluster can contain the following components:

      Doris

    • An analysis cluster contains the following components: Hadoop, Spark, HBase, Hive, Loader, Flink, Oozie, ZooKeeper, HetuEngine, Ranger, Tez, JobGateway, Guardian, and Doris.
    • A streaming cluster contains the following components: Kafka, Flume, ZooKeeper, and Ranger.
    • A hybrid cluster can contain the following components: Hadoop, Spark, HBase, Hive, Loader, Flink, Oozie, ZooKeeper, HetuEngine, Ranger, Tez, Kafka, Flume, JobGateway, and Guardian.
  • MRS 3.2.0-LTS.1 supports the following components:
    • An analysis cluster can contain the following components: Hadoop, Spark2x, HBase, Hive, Hue, Loader, Flink, Oozie, ZooKeeper, HetuEngine, Ranger, Tez, and Guardian.
    • A streaming cluster can contain the following components: Kafka, Flume, ZooKeeper, and Ranger.
    • A hybrid cluster can contain the following components: Hadoop, Spark2x, HBase, Hive, Hue, Loader, Flink, Oozie, ZooKeeper, HetuEngine, Ranger, Tez, Kafka, Flume, and Guardian.
    • A custom cluster can contain the following components: CDL, Hadoop, Spark2x, HBase, Hive, Hue, IoTDB, Loader, Kafka, Flume, Flink, Oozie, ZooKeeper, HetuEngine, Ranger, Tez, and ClickHouse, and Guardian.
  • MRS 3.1.5 supports the following components:
    • An analysis cluster can contain the following components: Hadoop, Spark2x, HBase, Hive, Hue, Flink, Oozie, ZooKeeper, Ranger, Tez, Impala, Presto, Kudu, Sqoop, and Guardian
    • A streaming cluster can contain the following components: Kafka, Flume, ZooKeeper, and Ranger.
    • A hybrid cluster can contain the following components: Hadoop, Spark2x, HBase, Hive, Hue, Flink, Oozie, ZooKeeper, Ranger, Tez, Impala, Presto, Kudu, Sqoop, Guardian, Kafka, and Flume.
    • A custom cluster can contain the following components: Hadoop, Spark2x, HBase, Hive, Hue, Kafka, Flume, Flink, Oozie, ZooKeeper, Ranger, Tez, Impala, Presto, ClickHouse, Kudu, Sqoop, and Guardian.
  • MRS 3.1.2-LTS.3 supports the following components:
    • An analysis cluster can contain the following components: Hadoop, Spark2x, HBase, Hive, Hue, Loader, Flink, Oozie, ZooKeeper, HetuEngine, Ranger, and Tez.
    • A streaming cluster can contain the following components: Kafka, Flume, ZooKeeper, and Ranger.
    • A hybrid cluster can contain the following components: Hadoop, Spark2x, HBase, Hive, Hue, Loader, Flink, Oozie, ZooKeeper, HetuEngine, Ranger, Tez, Kafka, and Flume.
    • A custom cluster can contain the following components: Hadoop, Spark2x, HBase, Hive, Hue, Loader, Kafka, Flume, Flink, Oozie, ZooKeeper, HetuEngine, Ranger, Tez, and ClickHouse.
  • MRS 3.1.0 supports the following components:
    • An analysis cluster can contain the following components: Hadoop, Spark2x, HBase, Hive, Hue, Flink, Oozie, ZooKeeper, Ranger, Tez, Impala, Presto, and Kudu.
    • A streaming cluster can contain the following components: Kafka, Flume, ZooKeeper, and Ranger.
    • A hybrid cluster can contain the following components: Hadoop, Spark2x, HBase, Hive, Hue, Flink, Oozie, ZooKeeper, Ranger, Tez, Impala, Presto, Kudu, Kafka, and Flume.
    • A custom cluster can contain the following components: Hadoop, Spark2x, HBase, Hive, Hue, Kafka, Flume, Flink, Oozie, ZooKeeper, Ranger, Tez, Impala, Presto, ClickHouse, and Kudu.
  • MRS 3.0.5 supports the following components:
    • An analysis cluster can contain the following components: Hadoop, Spark2x, HBase, Hive, Hue, Loader, Flink, Oozie, ZooKeeper, Ranger, Tez, Impala, Presto, Kudu, and Alluxio.
    • A streaming cluster can contain the following components: Kafka, Storm, Flume, ZooKeeper, and Ranger.
    • A hybrid cluster can contain the following components: Hadoop, Spark2x, HBase, Hive, Hue, Loader, Flink, Oozie, ZooKeeper, Ranger, Tez, Impala, Presto, Kudu, Alluxio, Kafka, Storm, and Flume.
    • A custom cluster can contain the following components: Hadoop, Spark2x, HBase, Hive, Hue, Loader, Kafka, Storm, Flume, Flink, Oozie, ZooKeeper, Ranger, Tez, Impala, Presto, ClickHouse, Kudu, and Alluxio.
  • MRS 2.1.0 supports the following components:
    • An analysis cluster can contain the following components: Presto, Hadoop, Spark, HBase, Hive, Hue, Loader, Tez, Impala, Kudu, and Flink.
    • A streaming cluster can contain the following components: Kafka, Storm, and Flume.
  • MRS 1.9.2 supports the following components:
    • An analysis cluster can contain the following components: Presto, Hadoop, Spark, HBase, OpenTSDB, Hive, Hue, Loader, Tez, Flink, Alluxio, and Ranger.
    • A streaming cluster can contain the following components: Kafka, KafkaManager, Storm, and Flume.

Constraints

  • You can log in to a cluster using either a password or a key pair.
  • To use the password mode, you need to configure the password of user root for accessing the cluster node, that is, cluster_master_secret.
  • To use the key pair mode, you need to configure the key pair name, that is, node_public_cert_name.
  • Disk parameters can be represented either by volume_type and volume_size, or by multi-disk parameters (master_data_volume_type, master_data_volume_size, master_data_volume_count, core_data_volume_type, core_data_volume_size, and core_data_volume_count).

Debugging

You can debug this API in API Explorer. Automatic authentication is supported. API Explorer can automatically generate sample SDK code and supports sample SDK code debugging.

Authorization Information

Each account has all the permissions required to call all APIs, but IAM users must be assigned the required permissions.

  • If you are using role/policy-based authorization, see Permissions Policies and Supported Actions for details on the required permissions.
  • If you are using identity policy-based authorization, no identity policy-based permissions are required for calling this API.

URI

POST /v1.1/{project_id}/run-job-flow
Table 2 URI parameters

Parameter

Mandatory

Type

Description

project_id

Yes

String

Definition

Project ID. For details about how to obtain the project ID, see Obtaining a Project ID.

Constraints

N/A

Range

The value must consist of 1 to 64 characters. Only letters and digits are allowed.

Default Value

N/A

Request Parameters

Table 3 Request parameters

Parameter

Mandatory

Type

Description

cluster_version

Yes

String

Definition

Cluster version, for example, MRS 3.1.0.

Constraints

N/A

Range

N/A

Default Value

N/A

cluster_name

Yes

String

Definition

Cluster name. It must be unique.

Constraints

N/A

Range

The value can contain 1 to 64 characters, including only letters, digits, underscores (_), and hyphens (-).

Default Value

N/A

master_node_num

No

Integer

Definition

Number of Master nodes.

Constraints

If cluster HA is enabled, set this parameter to 2. If cluster HA is disabled, set this parameter to 1. This parameter cannot be set to 1 in MRS 3.x.

Range

N/A

Default Value

N/A

core_node_num

No

Integer

Definition

Number of Core nodes. The default maximum number of core nodes is 500. If more than 500 core nodes are required, apply for a higher quota.

Constraints

N/A

Range

1-500

Default Value

N/A

billing_type

Yes

Integer

Definition

Cluster billing mode.

Constraints

N/A

Range

12: The cluster is billed on a pay-per-use basis. Only pay-per-use clusters can be created by calling APIs.

Default Value

N/A

data_center

Yes

String

Definition

The information about the region where the cluster is located. For details, see Endpoints.

Constraints

N/A

Range

N/A

Default Value

N/A

vpc

Yes

String

Definition

The name of the VPC where the subnet is located. Obtain the VPC name by performing the following operations on the VPC management console:

  1. Log in to the management console.
  2. Choose Virtual Private Cloud > My VPCs.
  3. On the Virtual Private Cloud page, obtain the VPC name from the list.

Constraints

N/A

Range

N/A

Default Value

N/A

master_node_size

No

String

Definition

Specifications of the Master node, for example, {ECS_FLAVOR_NAME}.linux.bigdata. {ECS_FLAVOR_NAME} can be c3.4xlare.2 or other flavors that are displayed on the MRS purchase page. The supported host specifications are determined by CPU, memory, and disk space. For details about instance specifications, see ECS Specifications Used by MRS and BMS Specifications Used by MRS. You are advised to obtain the specifications supported by the corresponding version in the corresponding region from the cluster creation page on the MRS console.

Constraints

N/A

Range

N/A

Default Value

N/A

core_node_size

No

String

Definition

Specifications of the Core node, for example, {ECS_FLAVOR_NAME}.linux.bigdata. {ECS_FLAVOR_NAME} can be c3.4xlare.2 or other flavors that are displayed on the MRS purchase page. For details about instance specifications, see ECS Specifications Used by MRS and BMS Specifications Used by MRS. You are advised to obtain the specifications supported by the corresponding version in the corresponding region from the cluster creation page on the MRS console.

Constraints

N/A

Range

N/A

Default Value

N/A

component_list

Yes

Array of component_list objects

Definition

The list of service components to be installed. For details about the parameters, see Table 4.

Constraints

N/A

Range

N/A

Default Value

N/A

available_zone_id

Yes

String

Definition

AZ ID. The following shows only some AZ IDs. You can obtain all AZ IDs by calling the API for querying AZ information.

Constraints

N/A

Range

  • CN-Hong Kong AZ1 (ap-southeast-1a): 8902e05a7ee04542a6a73246fddc46b0
  • CN-Hong Kong AZ2 (ap-southeast-1b): e9554f5c6fb84eeeb29ab766436b6454
  • AP-Bangkok AZ1 (ap-southeast-2a): 11d18bfa9d57488b8f96680013667546
  • AP-Bangkok AZ2 (ap-southeast-2b): 09d6a3cddbd643a5aa8837600c9af32c
  • AP-Singapore AZ1 (ap-southeast-3a): 82cc0d8877374316b669613539efd0d9
  • AP-Singapore AZ2 (ap-southeast-3b): 77394a8450e147779666771f796e9f03
  • AP-Singapore AZ3 (ap-southeast-3c): dba8d1bd3d9146659e2b5a38c09b19a4

Default Value

N/A

vpc_id

Yes

String

Definition

The ID of the VPC where the subnet is located.

Obtain the VPC ID by performing the following operations on the VPC management console:

  1. Log in to the management console.
  2. Choose Virtual Private Cloud > My VPCs.
  3. On the Virtual Private Cloud page, obtain the VPC ID from the list.

Constraints

N/A

Range

N/A

Default Value

N/A

subnet_id

Yes

String

Definition

Subnet ID Obtain the subnet ID by performing the following operations on the VPC management console:

  1. Log in to the management console.
  2. Choose Virtual Private Cloud > My VPCs.
  3. Locate the row that contains the target VPC and click the number in the Subnets column to view the subnet information.
  4. Click the subnet name to obtain the network ID.

Constraints

At least one of subnet_id and subnet_name must be configured. If the two parameters are configured but do not match the same subnet, the cluster fails to create. subnet_id is recommended.

Range

N/A

Default Value

N/A

subnet_name

Yes

String

Definition

Subnet name.

Obtain the subnet name by performing the following operations on the VPC management console:

  1. Log in to the management console.
  2. Choose Virtual Private Cloud > My VPCs.
  3. Locate the row that contains the target VPC and click the number in the Subnets column to obtain the subnet name.

Constraints

At least one of subnet_id and subnet_name must be configured. If the two parameters are configured but do not match the same subnet, the cluster fails to create. If only subnet_name is configured and subnets with the same name exist in the VPC, the first subnet name in the VPC is used when a cluster is created. subnet_id is recommended.

Range

N/A

Default Value

N/A

security_groups_id

No

String

Definition

The ID of the security group configured for the cluster.

  • If this parameter is left blank, MRS automatically creates a security group, whose name starts with mrs_{cluster_name}.
  • If this parameter is not left blank, a fixed security group is used to create a cluster. The transferred ID must be the security group ID owned by the current tenant. The security group must include an inbound rule in which all protocols and all ports are allowed and the source is the IP address of the specified node on the management plane.

Constraints

N/A

Range

N/A

Default Value

N/A

add_jobs

No

Array of add_jobs objects

Definition

Jobs can be submitted when a cluster is created. Currently, only one job can be created. For details about the parameters, see Table 5.

Constraints

There must be no more than 1 record.

Range

N/A

Default Value

N/A

volume_size

No

Integer

Definition

Data disk storage space of Master and Core nodes, in GB To increase the data storage capacity, you can add disks when creating a cluster. Select a proper disk storage space based on the following application scenarios:

  • Storage-compute decoupling: Data is stored in the OBS system. Costs of clusters are relatively low but computing performance is poor. The clusters can be deleted at any time. It is recommended when data computing is infrequently performed.
  • Storage-compute integration: Data is stored in the HDFS system. Costs of clusters are relatively high but computing performance is good. The clusters cannot be deleted in a short term. It is recommended when data computing is frequently performed.

Constraints

This parameter is not recommended. For details, see the description of the volume_type parameter.

Range

100-32000

Default Value

N/A

volume_type

No

String

Definition

The data disk storage type of master and core nodes. Currently, SATA, SAS, SSD, and GPSSD are supported. Disk parameters can be represented by volume_type and volume_size, or multi-disk parameters. If the volume_type and volume_size parameters coexist with the multi-disk parameters, the system reads the volume_type and volume_size parameters first. You are advised to use the multi-disk parameters.

Constraints

N/A

Range

  • SATA: common I/O
  • SAS: high I/O
  • SSD: ultra-high I/O
  • GPSSD: general-purpose SSD

Default Value

N/A

master_data_volume_type

No

String

Definition

This parameter is a multi-disk parameter, indicating the data disk storage type of the master node. Currently, SATA, SAS, SSD, and GPSSD are supported.

Constraints

N/A

Range

  • SATA: common I/O
  • SAS: high I/O
  • SSD: ultra-high I/O
  • GPSSD: general-purpose SSD

Default Value

N/A

master_data_volume_size

No

Integer

Definition

This parameter is a multi-disk parameter, indicating the data disk storage space of master nodes. To increase the data storage capacity, you can add disks when creating a cluster. You only need to pass in a number without the unit GB.

Constraints

N/A

Range

100-32000

Default Value

N/A

master_data_volume_count

No

Integer

Definition

This parameter is a multi-disk parameter, indicating the number of data disks of the master nodes.

Constraints

N/A

Range

The value can only be 1.

Default Value

1

core_data_volume_type

No

String

Definition

This parameter is a multi-disk parameter, indicating the data disk storage type of core nodes. Currently, SATA, SAS, SSD, and GPSSD are supported.

Constraints

N/A

Range

  • SATA: common I/O
  • SAS: high I/O
  • SSD: ultra-high I/O
  • GPSSD: general-purpose SSD

Default Value

N/A

core_data_volume_size

No

Integer

Definition

This parameter is a multi-disk parameter, indicating the data disk storage space of core nodes. To increase the data storage capacity, you can add disks when creating a cluster. You only need to pass in a number without the unit GB.

Constraints

N/A

Range

100-32000

Default Value

N/A

core_data_volume_count

No

Integer

Definition

This parameter is a multi-disk parameter, indicating the number of data disks of the core nodes.

Constraints

N/A

Range

1-20

Default Value

N/A

task_node_groups

No

Array of task_node_groups objects

Definition

The list of task nodes. For details about the parameters, see Table 6.

Constraints

There must be no more than 1 record.

Range

N/A

Default Value

N/A

bootstrap_scripts

No

Array of BootstrapScript objects

Definition

The Bootstrap action script information. For details about the parameters, see Table 8.

Constraints

N/A

Range

N/A

Default Value

N/A

node_public_cert_name

No

String

Definition

The name of a key pair. You can use a key pair to log in to a cluster node.

Constraints

If login_mode is set to 1, the request body contains the node_public_cert_name field.

Range

N/A

Default Value

N/A

cluster_admin_secret

No

String

Definition

Password of the MRS Manager administrator.

Constraints

N/A

Range

  • Must contain 8 to 26 characters.
  • Cannot be the username or the username spelled backwards.
  • Must contain every type of the following:
    • Lowercase letters
    • Uppercase letters
    • Numbers
    • Special characters (!@$%^-_=+[{}]:,./?)

Default Value

N/A

cluster_master_secret

No

String

Definition

The password of user root for logging in to a cluster node.

Constraints

If login_mode is set to 0, the request body contains the cluster_master_secret field.

Range

A password must meet the following complexity requirements:

  • Must be 8 to 26 characters long.
  • Must contain every type of the following: uppercase letters, lowercase letters, numbers, and special characters (!@$%^-_=+[{}]:,./?), but must not contain spaces.
  • Cannot be the username or the username spelled backwards.

Default Value

N/A

safe_mode

Yes

Integer

Definition

The running mode of an MRS cluster.

Constraints

N/A

Range

  • 0: normal cluster. In a normal cluster, Kerberos authentication is disabled, and users can use all functions provided by the cluster.
  • 1: security cluster. In a security cluster, Kerberos authentication is enabled, and common users cannot use the file management and job management functions of an MRS cluster or view cluster resource usage and the job records of Hadoop and Spark. To use more functions, the users must obtain the relevant permissions from the MRS Manager administrator.

Default Value

N/A

tags

No

Array of tag objects

Definition

The cluster tags. For details about the parameters, see Table 9.

Constraints

A maximum of 20 tags can be used in a cluster. The tag name (key) must be unique. The tag key and value can contain letters, digits, spaces, and special characters (_.:=+-@), but cannot start or end with a space or start with _sys_.

Range

N/A

Default Value

N/A

cluster_type

No

Integer

Definition

The cluster type. Currently, hybrid clusters cannot be created using APIs.

Constraints

N/A

Range

  • 0: analysis cluster
  • 1: streaming cluster

Default Value

0

log_collection

No

Integer

Definition

Whether to collect logs when cluster creation fails. The default value is 1, indicating that OBS buckets are created only for collecting logs when an MRS cluster fails to create.

Constraints

N/A

Range

  • 0: Do not collect logs.
  • 1: Collect logs.

Default Value

1

enterprise_project_id

No

String

Definition

Enterprise project ID When you create a cluster, associate the enterprise project ID with the cluster. The default value is 0, indicating the default enterprise project.

To obtain the enterprise project ID, see the id value in the enterprise_project field data structure table in "Querying the Enterprise Project List" in Enterprise Management API Reference.

Constraints

N/A

Range

N/A

Default Value

0

login_mode

No

Integer

Definition

Cluster login mode.

Constraints

  • If login_mode is set to 0, the request body contains the cluster_master_secret field.
  • If login_mode is set to 1, the request body contains the node_public_cert_name field.

Range

  • 0: password
  • 1: key pair

Default Value

1

node_groups

No

Array of NodeGroupV11 objects

Definition

List of nodes. For details about the parameters, see Table 10.

Constraints

Configure either this parameter or the following parameters:

master_node_num, master_node_size, core_node_num, core_node_size, master_data_volume_type, master_data_volume_size, master_data_volume_count, core_data_volume_type, core_data_volume_size, core_data_volume_count, volume_type, volume_size, task_node_groups

Range

N/A

Default Value

N/A
Table 4 ComponentAmbV11

Parameter

Mandatory

Type

Description

component_name

Yes

String

Definition

Component name. For details, see the component information in Table 1.

Constraints

N/A

Range

The value can contain 1 to 64 characters, including only letters, digits, underscores (_), and hyphens (-).

Default Value

N/A
Table 5 AddJobsReqV11

Parameter

Mandatory

Type

Description

job_type

Yes

Integer

Definition

Job type code.

Constraints

N/A

Range

  • 1: MapReduce
  • 2: Spark
  • 3: Hive Script
  • 4: HiveQL (not supported currently)
  • 5: DistCp for importing and exporting data (not supported currently)
  • 6: Spark Script
  • 7: Spark SQL for submitting Spark SQL statements (not supported currently)
    NOTE:

    Spark and Hive jobs can be created only in clusters that where Spark and Hive are installed.

Default Value

N/A

job_name

Yes

String

Definition

Job name.

Constraints

N/A

Range

The value can contain 1 to 64 characters. Only letters, digits, underscores (_), and hyphens (-) are allowed. Job names can be the same across different jobs, but you are not advised to set them the same.

Default Value

N/A

jar_path

No

String

Definition

Path of the .jar file or .sql file to be executed.

Constraints

N/A

Range

The value must meet the following requirements:

  • The value contains a maximum of 1,023 characters. It cannot contain special characters (;|&>,<'$) and cannot be left blank or all spaces.
  • Files can be stored in HDFS or OBS. The path varies depending on the file system.
    • OBS: The path must start with s3a://. Files or programs encrypted by KMS are not supported.
    • HDFS: The path starts with a slash (/).
  • Spark Script must end with .sql while MapReduce and Spark Jar must end with .jar. sql and jar are case-insensitive.

Default Value

N/A

arguments

No

String

Definition

The key parameter for program execution. The parameter is specified by the function of the user's program. MRS is only responsible for loading the parameter.

Constraints

N/A

Range

The parameter can contain 0 to 150,000 characters, but special characters (;|&>'<$) are not allowed.

Default Value

N/A

input

No

String

Definition

The data input path.

Files can be stored in HDFS or OBS. The path varies depending on the file system.
  • OBS: The path must start with s3a://. Files or programs encrypted by KMS are not supported.
  • HDFS: The path starts with a slash (/).

Constraints

N/A

Range

The value can contain 0 to 1,023 characters, but special characters (;|&>'<$) are not allowed.

Default Value

N/A

output

No

String

Definition

The data output path.

Files can be stored in HDFS or OBS. The path varies depending on the file system.
  • OBS: The path must start with s3a://.
  • HDFS: The path starts with a slash (/).

If the specified path does not exist, the system will automatically create it.

Constraints

N/A

Range

The value can contain 0 to 1,023 characters, but special characters (;|&>'<$) are not allowed.

Default Value

N/A

job_log

No

String

Definition

The path for storing job logs that record job running status.

Files can be stored in HDFS or OBS. The path varies depending on the file system.
  • OBS: The path must start with s3a://.
  • HDFS: The path starts with a slash (/).

Constraints

N/A

Range

The value can contain 0 to 1,023 characters, but special characters (;|&>'<$) are not allowed.

Default Value

N/A

shutdown_cluster

No

Boolean

Definition

Whether to delete the cluster after the job execution is complete.

Constraints

N/A

Range

  • true: Delete the cluster after job execution is complete.
  • false: Do not delete the cluster after the job execution is complete.

Default Value

N/A

file_action

No

String

Definition

The action to be performed on a file.

Constraints

N/A

Range

  • import: Import data.
  • export: Export data.

Default Value

N/A

submit_job_once_cluster_run

Yes

Boolean

Definition

Whether to submit a job when creating a cluster. Set it to true.

Constraints

N/A

Range

  • true: Submit a job during cluster creation.
  • false: Submit a job after the cluster is created.

Default Value

N/A

hql

No

String

Definition

The HQL script statement.

Constraints

N/A

Range

N/A

Default Value

N/A

hive_script_path

No

String

Definition

SQL program path. This parameter is required by Spark Script and Hive Script jobs only.

Constraints

N/A

Range

The value must meet the following requirements:

  • The value contains a maximum of 1,023 characters. It cannot contain special characters (;|&><'$) and cannot be left blank or all spaces.
  • Files can be stored in HDFS or OBS. The path varies depending on the file system.
    • OBS: The path must start with s3a://. Files or programs encrypted by KMS are not supported.
    • HDFS: The path starts with a slash (/).
  • Ends with .sql. sql is case-insensitive.

Default Value

N/A
Table 6 TaskNodeGroup

Parameter

Mandatory

Type

Description

node_num

Yes

Integer

Definition

Number of Task nodes.

Constraints

The total number of Core and Task nodes cannot exceed 500.

Range

0-500

Default Value

N/A

node_size

Yes

String

Definition

Specifications of the Task node, for example, {ECS_FLAVOR_NAME}.linux.bigdata. {ECS_FLAVOR_NAME} can be c3.4xlare.2 or other flavors that are displayed on the MRS purchase page. For details about instance specifications, see ECS Specifications Used by MRS and BMS Specifications Used by MRS.

Obtain the instance specifications of the corresponding version in the corresponding region from the cluster creation page of the MRS management console.

Constraints

N/A

Range

N/A

Default Value

N/A

data_volume_type

Yes

String

Definition

Data disk storage type of the Task node. Supported types include SATA, SAS, and SSD.

Constraints

N/A

Range

  • SATA: common I/O
  • SAS: high I/O
  • SSD: ultra-high I/O
  • GPSSD: general-purpose SSD

Default Value

N/A

data_volume_count

Yes

Integer

Definition

Number of data disks of a Task node.

Constraints

N/A

Range

0-20

Default Value

N/A

data_volume_size

Yes

Integer

Definition

Data disk storage space of a Task node. You only need to pass in a number without the unit GB.

Constraints

N/A

Range

100-32000

Default Value

N/A

auto_scaling_policy

No

auto_scaling_policy object

Definition

The auto scaling policy. For details, see Table 7.

Constraints

N/A

Range

N/A

Default Value

N/A
Table 7 AutoScalingPolicy

Parameter

Mandatory

Type

Description

auto_scaling_enable

Yes

Boolean

Definition

Whether to enable the auto scaling policy.

Constraints

N/A

Range

  • true: Enable the auto scaling rule.
  • false: Disable the autoscaling rule.

Default Value

N/A

min_capacity

Yes

Integer

Definition

The minimum number of nodes reserved in the node group.

Constraints

N/A

Range

0-500

Default Value

N/A

max_capacity

Yes

Integer

Definition

The maximum number of nodes in the node group.

Constraints

N/A

Range

0-500

Default Value

N/A

resources_plans

No

Array of resources_plan objects

Definition

The resource plan list. For details, see Table 11. If this parameter is left blank, the resource plan is disabled.

Constraints

When auto_scaling_enable is set to true, either this parameter or rules must be configured. There must be no more than 5 records.

Range

N/A

Default Value

N/A

exec_scripts

No

Array of scale_script objects

Definition

The list of custom scaling automation scripts. For details, see Table 14. If this parameter is left blank, a hook script is disabled.

Constraints

The number of records cannot exceed 10.

Range

N/A

Default Value

N/A

rules

No

Array of rules objects

Definition

The list of auto scaling rules. For details, see Table 12.

Constraints

When auto_scaling_enable is set to true, either this parameter or resources_plans must be configured. The number of records cannot exceed 10.

Range

N/A

Default Value

N/A
Table 8 BootstrapScript

Parameter

Mandatory

Type

Description

name

Yes

String

Definition

Name of a bootstrap action script.

Constraints

N/A

Range

The names of bootstrap action scripts in the same cluster must be unique. The value can contain 1 to 64 characters, including only letters, digits, underscores (_), and hyphens (-), and cannot start with a space.

Default Value

N/A

uri

Yes

String

Definition

The path of a Bootstrap action script. Set this parameter to an OBS bucket path or a local VM path.

  • OBS bucket path: Enter a script path. For example, enter the path of the public sample script provided by MRS. Example: s3a://bootstrap/presto/presto-install.sh. If dualroles is installed, the parameter of the presto-install.sh script is dualroles. If worker is installed, the parameter of the presto-install.sh script is worker. Based on the Presto usage habit, you are advised to install dualroles on the active master nodes and worker on the Core nodes.
  • Local VM path: Enter a script path. The script path must start with a slash (/) and end with .sh.

Constraints

N/A

Range

N/A

Default Value

N/A

parameters

No

String

Definition

The bootstrap action script parameters.

Constraints

N/A

Range

N/A

Default Value

N/A

nodes

Yes

Array of strings

Definition

The type of a node where the bootstrap action script is executed. The value can be Master, Core, or Task.

Constraints

The node type must be represented in lowercase letters.

Range

N/A

Default Value

N/A

active_master

No

Boolean

Definition

Whether the bootstrap action script runs only on active master nodes.

Constraints

N/A

Range

  • true: The bootstrap action script runs only on active Master nodes.
  • false: The bootstrap action script can run on all Master nodes.

Default Value

false

before_component_start

No

Boolean

Definition

Time when the bootstrap action script is executed. Currently, the following two options are available: Before component start and After component start

Constraints

N/A

Range

  • true: The bootstrap action script is executed before the component starts.
  • false: The bootstrap action script is executed after the component starts.

Default Value

false

fail_action

Yes

String

Definition

Whether to continue executing subsequent scripts and creating a cluster after the Bootstrap action script fails to be executed.

You are advised to set this parameter to continue in the commissioning phase so that the cluster can continue to be installed and started no matter whether the bootstrap action is successful.

Constraints

N/A

Range

  • continue: Continue to execute subsequent scripts.
  • errorout: Stop the action.

Default Value

errorout

start_time

No

Long

Definition

The execution time of one bootstrap action script.

Constraints

N/A

Range

N/A

Default Value

N/A

state

No

String

Definition

The running status of one bootstrap action script.

Constraints

N/A

Range

  • PENDING: The script is suspended.
  • IN_PROGRESS: The script is being processed.
  • SUCCESS
  • FAILURE: The script fails to be executed.

Default Value

N/A

action_stages

No

Array of strings

Definition

Select the time when the bootstrap action script is executed.

  • BEFORE_COMPONENT_FIRST_START: before initial component starts
  • AFTER_COMPONENT_FIRST_START: after initial component starts
  • BEFORE_SCALE_IN: before scale-in
  • AFTER_SCALE_IN: after scale-in
  • BEFORE_SCALE_OUT: before scale-out
  • AFTER_SCALE_OUT: after scale-out

Constraints

N/A

Range

N/A

Default Value

N/A
Table 9 Tag

Parameter

Mandatory

Type

Description

key

Yes

String

Definition

Tag key.

Constraints

N/A

Range

  • The value can contain a maximum of 128 characters and cannot be an empty string.
  • The tag key of a resource must be unique.
  • A tag key can contain letters, digits, spaces, and special characters _.:=+-@, but cannot start or end with a space or start with _sys_.

Default Value

N/A

value

Yes

String

Definition

Tag value.

Constraints

N/A

Range

  • The value can contain a maximum of 255 characters and can be an empty string.
  • The value can contain letters, digits, spaces, and special characters _.:=+-@, but cannot start or end with a space or start with _sys_.

Default Value

N/A
Table 10 NodeGroupV11

Parameter

Mandatory

Type

Description

group_name

Yes

String

Definition

The node group name.

Constraints

N/A

Range

  • master_node_default_group
  • core_node_analysis_group
  • core_node_streaming_group
  • task_node_analysis_group
  • task_node_streaming_group

Default Value

N/A

node_num

Yes

Integer

Definition

Number of nodes.

Constraints

The total number of Core and Task nodes cannot exceed 500.

Range

0-500

Default Value

N/A

node_size

Yes

String

Definition

Specifications of the node, for example, {ECS_FLAVOR_NAME}.linux.bigdata. {ECS_FLAVOR_NAME} can be c3.4xlare.2 or other flavors that are displayed on the MRS purchase page. The host specifications supported by MRS are determined by CPU, memory, and disk space. For details about instance specifications, see ECS Specifications Used by MRS and BMS Specifications Used by MRS. You are advised to obtain the specifications supported by the corresponding version in the corresponding region from the cluster creation page on the MRS console.

Constraints

N/A

Range

N/A

Default Value

N/A

root_volume_size

No

String

Definition

System disk storage size of a node, in GB.

Constraints

N/A

Range

N/A

Default Value

N/A

root_volume_type

No

String

Definition

System disk storage type of a node. Supported types include SATA, SAS, and SSD.

Constraints

N/A

Range

  • SATA: common I/O
  • SAS: high I/O
  • SSD: ultra-high I/O
  • GPSSD: general-purpose SSD

Default Value

N/A

data_volume_type

No

String

Definition

Data disk storage type of nodes. Supported types include SATA, SAS, and SSD.

Constraints

N/A

Range

  • SATA: common I/O
  • SAS: high I/O
  • SSD: ultra-high I/O
  • GPSSD: general-purpose SSD

Default Value

N/A

data_volume_count

No

Integer

Definition

Number of data disks of a node.

Constraints

N/A

Range

0-20

Default Value

N/A

data_volume_size

No

Integer

Definition

Data disk storage space of a node.

Unit: GB.

Constraints

N/A

Range

100-32000

Default Value

N/A

auto_scaling_policy

No

auto_scaling_policy object

Definition

The auto scaling policy.

Constraints

The auto scaling rule information. This parameter is available only when group_name is set to task_node_analysis_group or task_node_streaming_group.

For details about the parameters, see Table 7.

Range

N/A

Default Value

N/A
Table 11 ResourcesPlan

Parameter

Mandatory

Type

Description

period_type

Yes

String

Definition

Cycle type of a resource plan. This parameter can be set to daily only.

Constraints

N/A

Range

  • daily: Charges are calculated by day.

Default Value

N/A

start_time

Yes

String

Definition

The start time of a resource plan. The value is in the format of hour:minute, indicating that the time ranges from 00:00 to 23:59.

Constraints

N/A

Range

N/A

Default Value

N/A

end_time

Yes

String

Definition

End time of a resource plan. The format is the same as that of start_time.

Constraints

The value cannot be earlier than the start_time, and the interval between start_time and start_time cannot be less than 30 minutes.

Range

N/A

Default Value

N/A

min_capacity

Yes

Integer

Definition

Minimum number of the preserved nodes in a node group in a resource plan.

Constraints

N/A

Range

0-500

Default Value

N/A

max_capacity

Yes

Integer

Definition

Maximum number of the preserved nodes in a node group in a resource plan.

Constraints

N/A

Range

0-500

Default Value

N/A
Table 12 Rule

Parameter

Mandatory

Type

Description

name

Yes

String

Definition

Name of an auto scaling rule.

Constraints

N/A

Range

It contains only 1 to 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed.

Rule names must be unique in a node group.

Default Value

N/A

description

No

String

Definition

Description about an auto scaling rule.

Constraints

N/A

Range

It contains a maximum of 1024 characters.

Default Value

N/A

adjustment_type

Yes

String

Definition

Auto scaling rule adjustment type.

Constraints

N/A

Range

  • scale_out: cluster scale-out
  • scale_in: cluster scale-in

Default Value

N/A

cool_down_minutes

Yes

Integer

Definition

Cluster cooling time after an auto scaling rule is triggered, when no auto scaling operation is performed. The unit is minute.

Constraints

N/A

Range

The value ranges from 0 to 10080. 10080 indicates the number of minutes in a week.

Default Value

N/A

scaling_adjustment

Yes

Integer

Definition

Number of nodes that can be adjusted once.

Constraints

N/A

Range

1-100

Default Value

N/A

trigger

Yes

Trigger object

Definition

Condition for triggering a rule. For details, see Table 13.

Constraints

N/A

Range

N/A

Default Value

N/A
Table 13 Trigger

Parameter

Mandatory

Type

Description

metric_name

Yes

String

Definition

Metric name. This triggering condition makes a judgment according to the value of the metric.

For details about metric names, see Configuring Auto Scaling for an MRS Cluster.

Constraints

N/A

Range

A metric name contains a maximum of 64 characters.

Default Value

N/A

metric_value

Yes

String

Definition

Metric threshold to trigger a rule The value must be an integer or a number with two decimal places.

Constraints

N/A

Range

Only integers or numbers with two decimal places are allowed.

Default Value

N/A

comparison_operator

No

String

Definition

Metric judgment logic operator.

Constraints

N/A

Range

  • LT: less than
  • GT: greater than
  • LTOE: less than or equal to
  • GTOE: greater than or equal to

Default Value

N/A

evaluation_periods

Yes

Integer

Definition

The number of consecutive five-minute periods, during which a metric threshold is reached.

Constraints

N/A

Range

1-200

Default Value

N/A
Table 14 ScaleScript

Parameter

Mandatory

Type

Description

name

Yes

String

Definition

Names of custom scaling automation scripts.

Constraints

N/A

Range

The names in the same cluster must be unique. The value can contain 1 to 64 characters, including only digits, letters, spaces, hyphens (-), and underscores (_), and cannot start with a space.

Default Value

N/A

uri

Yes

String

Definition

Path of a custom automation script. Set this parameter to an OBS bucket path or a local VM path.

  • OBS bucket path: Enter a script path manually, for example, s3a://XXX/scale.sh.
  • Local VM path: Enter a script path. The script path must start with a slash (/) and end with .sh.

Constraints

N/A

Range

N/A

Default Value

N/A

parameters

No

String

Definition

Parameters of a custom automation script. Separate multiple parameters by spaces. The following predefined parameters can be transferred:

  • ${mrs_scale_node_num}: Number of the nodes to be added or removed
  • ${mrs_scale_type}: Scaling type. The value can be scale_out or scale_in.
  • ${mrs_scale_node_hostnames}: Host names of the nodes to be added or removed
  • ${mrs_scale_node_ips}: IP addresses of the nodes to be added or removed
  • ${mrs_scale_rule_name}: Name of the rule that triggers auto scaling

Other user-defined parameters are used in the same way as those of common shell scripts. Parameters are separated by space.

Constraints

N/A

Range

N/A

Default Value

N/A

nodes

Yes

Array of strings

Definition

Type of a node where the custom automation script is executed. The node type can be Master, Core, or Task.

Constraints

N/A

Range

N/A

Default Value

N/A

active_master

No

Boolean

Definition

Whether the custom automation script runs only on the active master node.

Constraints

N/A

Range

  • true: The custom automation script runs only on the active Master nodes.
  • false: The custom automation script can run on all Master nodes.

Default Value

false

action_stage

Yes

String

Definition

Time when a script is executed.

Constraints

N/A

Range

  • before_scale_out: before scale-out
  • before_scale_in: before scale-in
  • after_scale_out: after scale-out
  • after_scale_in: after scale-in

Default Value

N/A

fail_action

Yes

String

Definition

Whether to continue to execute subsequent scripts and create a cluster after the custom automation script fails to be executed. You are advised to set this parameter to continue in the commissioning phase so the cluster can continue to be installed and started no matter whether the custom automation script is executed successfully. The scale-in operation cannot be undone. fail_action must be set to continue for the scripts that are executed after scale-in.

Constraints

N/A

Range

  • continue: Continue to execute subsequent scripts.
  • errorout: Stop the action.

Default Value

N/A

Response Parameters

Status code: 200

Table 15 Response body parameters

Parameter

Type

Description

cluster_id

String

Definition

Cluster ID, which is returned by the system after the cluster is created.

Range

N/A

result

Boolean

Definition

Operation result

Range

  • true: The operation is successful.
  • false: The operation failed.

msg

String

Definition

System message, which can be empty.

Range

N/A

Example Request

  • Use the node_groups parameter group to create a cluster with the HA function enabled. The cluster version is MRS 3.2.0-LTS.1. 
    POST https://{endpoint}/v1.1/{project_id}/run-job-flow

{
  "billing_type" : 12,
  "data_center" : "",
  "available_zone_id" : "0e7a368b6c54493e94ad32666b47e23e",
  "cluster_name" : "mrs_HEbK",
  "cluster_version" : "MRS 3.2.0-LTS.1",
  "safe_mode" : 0,
  "cluster_type" : 0,
  "component_list" : [ {
    "component_name" : "Hadoop"
  }, {
    "component_name" : "Spark2x"
  }, {
    "component_name" : "HBase"
  }, {
    "component_name" : "Hive"
  }, {
    "component_name" : "Zookeeper"
  }, {
    "component_name" : "Tez"
  }, {
    "component_name" : "Hue"
  }, {
    "component_name" : "Loader"
  }, {
    "component_name" : "Flink"
  } ],
  "vpc" : "vpc-4b1c",
  "vpc_id" : "4a365717-67be-4f33-80c5-98e98a813af8",
  "subnet_id" : "67984709-e15e-4e86-9886-d76712d4e00a",
  "subnet_name" : "subnet-4b44",
  "security_groups_id" : "4820eace-66ad-4f2c-8d46-cf340e3029dd",
  "enterprise_project_id" : "0",
  "tags" : [ {
    "key" : "key1",
    "value" : "value1"
  }, {
    "key" : "key2",
    "value" : "value2"
  } ],
  "node_groups" : [ {
    "group_name" : "master_node_default_group",
    "node_num" : 2,
    "node_size" : "s3.xlarge.2.linux.bigdata",
    "root_volume_size" : 480,
    "root_volume_type" : "SATA",
    "data_volume_type" : "SATA",
    "data_volume_count" : 1,
    "data_volume_size" : 600
  }, {
    "group_name" : "core_node_analysis_group",
    "node_num" : 3,
    "node_size" : "s3.xlarge.2.linux.bigdata",
    "root_volume_size" : 480,
    "root_volume_type" : "SATA",
    "data_volume_type" : "SATA",
    "data_volume_count" : 1,
    "data_volume_size" : 600
  }, {
    "group_name" : "task_node_analysis_group",
    "node_num" : 2,
    "node_size" : "s3.xlarge.2.linux.bigdata",
    "root_volume_size" : 480,
    "root_volume_type" : "SATA",
    "data_volume_type" : "SATA",
    "data_volume_count" : 0,
    "data_volume_size" : 600,
    "auto_scaling_policy" : {
      "auto_scaling_enable" : true,
      "min_capacity" : 1,
      "max_capacity" : "3",
      "resources_plans" : [ {
        "period_type" : "daily",
        "start_time" : "9:50",
        "end_time" : "10:20",
        "min_capacity" : 2,
        "max_capacity" : 3
      }, {
        "period_type" : "daily",
        "start_time" : "10:20",
        "end_time" : "12:30",
        "min_capacity" : 0,
        "max_capacity" : 2
      } ],
      "exec_scripts" : [ {
        "name" : "before_scale_out",
        "uri" : "s3a://XXX/zeppelin_install.sh",
        "parameters" : "${mrs_scale_node_num} ${mrs_scale_type} xxx",
        "nodes" : [ "master", "core", "task" ],
        "active_master" : "true",
        "action_stage" : "before_scale_out",
        "fail_action" : "continue"
      }, {
        "name" : "after_scale_out",
        "uri" : "s3a://XXX/storm_rebalance.sh",
        "parameters" : "${mrs_scale_node_hostnames} ${mrs_scale_node_ips}",
        "nodes" : [ "master", "core", "task" ],
        "active_master" : "true",
        "action_stage" : "after_scale_out",
        "fail_action" : "continue"
      } ],
      "rules" : [ {
        "name" : "default-expand-1",
        "adjustment_type" : "scale_out",
        "cool_down_minutes" : 5,
        "scaling_adjustment" : 1,
        "trigger" : {
          "metric_name" : "YARNMemoryAvailablePercentage",
          "metric_value" : "25",
          "comparison_operator" : "LT",
          "evaluation_periods" : 10
        }
      }, {
        "name" : "default-shrink-1",
        "adjustment_type" : "scale_in",
        "cool_down_minutes" : 5,
        "scaling_adjustment" : 1,
        "trigger" : {
          "metric_name" : "YARNMemoryAvailablePercentage",
          "metric_value" : "70",
          "comparison_operator" : "GT",
          "evaluation_periods" : 10
        }
      } ]
    }
  } ],
  "login_mode" : 1,
  "cluster_master_secret" : "",
  "cluster_admin_secret" : "",
  "log_collection" : 1,
  "add_jobs" : [ {
    "job_type" : 1,
    "job_name" : "tenji111",
    "jar_path" : "s3a://bigdata/program/hadoop-mapreduce-examples-2.7.2.jar",
    "arguments" : "wordcount",
    "input" : "s3a://bigdata/input/wd_1k/",
    "output" : "s3a://bigdata/output/",
    "job_log" : "s3a://bigdata/log/",
    "shutdown_cluster" : true,
    "file_action" : "",
    "submit_job_once_cluster_run" : true,
    "hql" : "",
    "hive_script_path" : ""
  } ],
  "bootstrap_scripts" : [ {
    "name" : "Modify os config",
    "uri" : "s3a://XXX/modify_os_config.sh",
    "parameters" : "param1 param2",
    "nodes" : [ "master", "core", "task" ],
    "active_master" : "false",
    "before_component_start" : "true",
    "start_time" : "1667892101",
    "state" : "IN_PROGRESS",
    "fail_action" : "continue",
    "action_stages" : [ "BEFORE_COMPONENT_FIRST_START", "BEFORE_SCALE_IN" ]
  }, {
    "name" : "Install zeppelin",
    "uri" : "s3a://XXX/zeppelin_install.sh",
    "parameters" : "",
    "nodes" : [ "master" ],
    "active_master" : "true",
    "before_component_start" : "false",
    "start_time" : "1667892101",
    "state" : "IN_PROGRESS",
    "fail_action" : "continue",
    "action_stages" : [ "AFTER_SCALE_IN", "AFTER_SCALE_OUT" ]
  } ]
}
  • Create a cluster with the HA function enabled without using the node_groups parameter group. The cluster version is MRS 3.2.0-LTS.1. 
    POST https://{endpoint}/v1.1/{project_id}/run-job-flow

{
  "billing_type" : 12,
  "data_center" : "",
  "master_node_num" : 2,
  "master_node_size" : "s3.2xlarge.2.linux.bigdata",
  "core_node_num" : 3,
  "core_node_size" : "s3.2xlarge.2.linux.bigdata",
  "available_zone_id" : "0e7a368b6c54493e94ad32666b47e23e", 
  "cluster_name" : "newcluster",
  "vpc" : "vpc1",
  "vpc_id" : "5b7db34d-3534-4a6e-ac94-023cd36aaf74",
  "subnet_id" : "815bece0-fd22-4b65-8a6e-15788c99ee43",
  "subnet_name" : "subnet",
  "security_groups_id" : "845bece1-fd22-4b45-7a6e-14338c99ee43",
  "tags" : [ {
    "key" : "key1",
    "value" : "value1"
  }, {
    "key" : "key2",
    "value" : "value2"
  } ],
  "cluster_version" : "MRS 3.2.0-LTS.1",
  "cluster_type" : 0,
  "master_data_volume_type" : "SATA",
  "master_data_volume_size" : 600,
  "master_data_volume_count" : 1,
  "core_data_volume_type" : "SATA",
  "core_data_volume_size" : 600,
  "core_data_volume_count" : 2,
  "node_public_cert_name" : "SSHkey-bba1",
  "safe_mode" : 0,
  "log_collection" : 1,
  "task_node_groups" : [ {
    "node_num" : 2,
    "node_size" : "s3.xlarge.2.linux.bigdata",
    "data_volume_type" : "SATA",
    "data_volume_count" : 1,
    "data_volume_size" : 600,
    "auto_scaling_policy" : {
      "auto_scaling_enable" : true,
      "min_capacity" : 1,
      "max_capacity" : "3",
      "resources_plans" : [ {
        "period_type" : "daily",
        "start_time" : "9: 50",
        "end_time" : "10: 20",
        "min_capacity" : 2,
        "max_capacity" : 3
      }, {
        "period_type" : "daily",
        "start_time" : "10: 20",
        "end_time" : "12: 30",
        "min_capacity" : 0,
        "max_capacity" : 2
      } ],
      "exec_scripts" : [ {
        "name" : "before_scale_out",
        "uri" : "s3a: //XXX/zeppelin_install.sh",
        "parameters" : "${mrs_scale_node_num}${mrs_scale_type}xxx",
        "nodes" : [ "master", "core", "task" ],
        "active_master" : "true",
        "action_stage" : "before_scale_out",
        "fail_action" : "continue"
      }, {
        "name" : "after_scale_out",
        "uri" : "s3a: //XXX/storm_rebalance.sh",
        "parameters" : "${mrs_scale_node_hostnames}${mrs_scale_node_ips}",
        "nodes" : [ "master", "core", "task" ],
        "active_master" : "true",
        "action_stage" : "after_scale_out",
        "fail_action" : "continue"
      } ],
      "rules" : [ {
        "name" : "default-expand-1",
        "adjustment_type" : "scale_out",
        "cool_down_minutes" : 5,
        "scaling_adjustment" : 1,
        "trigger" : {
          "metric_name" : "YARNMemoryAvailablePercentage",
          "metric_value" : "25",
          "comparison_operator" : "LT",
          "evaluation_periods" : 10
        }
      }, {
        "name" : "default-shrink-1",
        "adjustment_type" : "scale_in",
        "cool_down_minutes" : 5,
        "scaling_adjustment" : 1,
        "trigger" : {
          "metric_name" : "YARNMemoryAvailablePercentage",
          "metric_value" : "70",
          "comparison_operator" : "GT",
          "evaluation_periods" : 10
        }
      } ]
    }
  } ],
  "component_list" : [ {
    "component_name" : "Hadoop"
  }, {
    "component_name" : "Spark"
  }, {
    "component_name" : "HBase"
  }, {
    "component_name" : "Hive"
  } ],
  "add_jobs" : [ {
    "job_type" : 1,
    "job_name" : "tenji111",
    "jar_path" : "s3a: //bigdata/program/hadoop-mapreduce-examples-2.7.2.jar",
    "arguments" : "wordcount",
    "input" : "s3a: //bigdata/input/wd_1k/",
    "output" : "s3a: //bigdata/output/",
    "job_log" : "s3a: //bigdata/log/",
    "shutdown_cluster" : true,
    "file_action" : "",
    "submit_job_once_cluster_run" : true,
    "hql" : "",
    "hive_script_path" : ""
  } ],
  "bootstrap_scripts" : [ {
    "name" : "Modifyosconfig",
    "uri" : "s3a: //XXX/modify_os_config.sh",
    "parameters" : "param1param2",
    "nodes" : [ "master", "core", "task" ],
    "active_master" : "false",
    "before_component_start" : "true",
    "start_time" : "1667892101",
    "state" : "IN_PROGRESS",
    "fail_action" : "continue",
    "action_stages" : [ "BEFORE_COMPONENT_FIRST_START", "BEFORE_SCALE_IN" ]
  }, {
    "name" : "Installzeppelin",
    "uri" : "s3a: //XXX/zeppelin_install.sh",
    "parameters" : "",
    "nodes" : [ "master" ],
    "active_master" : "true",
    "before_component_start" : "false",
    "start_time" : "1667892101",
    "state" : "IN_PROGRESS",
    "fail_action" : "continue",
    "action_stages" : [ "AFTER_SCALE_IN", "AFTER_SCALE_OUT" ]
  } ]
}
  • Use the node_groups parameter group to create a cluster with the HA function disabled. The cluster version is MRS 3.2.0-LTS.1. 
    POST https://{endpoint}/v1.1/{project_id}/run-job-flow

{
  "billing_type" : 12,
  "data_center" : "",
  "available_zone_id" : "0e7a368b6c54493e94ad32666b47e23e",
  "cluster_name" : "mrs_HEbK",
  "cluster_version" : "MRS 3.2.0-LTS.1",
  "safe_mode" : 0,
  "cluster_type" : 0,
  "component_list" : [ {
    "component_name" : "Hadoop"
  }, {
    "component_name" : "Spark2x"
  }, {
    "component_name" : "HBase"
  }, {
    "component_name" : "Hive"
  }, {
    "component_name" : "Zookeeper"
  }, {
    "component_name" : "Tez"
  }, {
    "component_name" : "Hue"
  }, {
    "component_name" : "Loader"
  }, {
    "component_name" : "Flink"
  } ],
  "vpc" : "vpc-4b1c",
  "vpc_id" : "4a365717-67be-4f33-80c5-98e98a813af8",
  "subnet_id" : "67984709-e15e-4e86-9886-d76712d4e00a",
  "subnet_name" : "subnet-4b44",
  "security_groups_id" : "4820eace-66ad-4f2c-8d46-cf340e3029dd",
  "enterprise_project_id" : "0",
  "tags" : [ {
    "key" : "key1",
    "value" : "value1"
  }, {
    "key" : "key2",
    "value" : "value2"
  } ],
  "node_groups" : [ {
    "group_name" : "master_node_default_group",
    "node_num" : 1,
    "node_size" : "s3.xlarge.2.linux.bigdata",
    "root_volume_size" : 480,
    "root_volume_type" : "SATA",
    "data_volume_type" : "SATA",
    "data_volume_count" : 1,
    "data_volume_size" : 600
  }, {
    "group_name" : "core_node_analysis_group",
    "node_num" : 1,
    "node_size" : "s3.xlarge.2.linux.bigdata",
    "root_volume_size" : 480,
    "root_volume_type" : "SATA",
    "data_volume_type" : "SATA",
    "data_volume_count" : 1,
    "data_volume_size" : 600
  } ],
  "login_mode" : 1,
  "cluster_master_secret" : "",
  "cluster_admin_secret" : "",
  "log_collection" : 1,
  "add_jobs" : [ {
    "job_type" : 1,
    "job_name" : "tenji111",
    "jar_path" : "s3a://bigdata/program/hadoop-mapreduce-examples-2.7.2.jar",
    "arguments" : "wordcount",
    "input" : "s3a://bigdata/input/wd_1k/",
    "output" : "s3a://bigdata/output/",
    "job_log" : "s3a://bigdata/log/",
    "shutdown_cluster" : true,
    "file_action" : "",
    "submit_job_once_cluster_run" : true,
    "hql" : "",
    "hive_script_path" : ""
  } ],
  "bootstrap_scripts" : [ {
    "name" : "Modify os config",
    "uri" : "s3a://XXX/modify_os_config.sh",
    "parameters" : "param1 param2",
    "nodes" : [ "master", "core", "task" ],
    "active_master" : "false",
    "before_component_start" : "true",
    "start_time" : "1667892101",
    "state" : "IN_PROGRESS",
    "fail_action" : "continue",
    "action_stages" : [ "BEFORE_COMPONENT_FIRST_START", "BEFORE_SCALE_IN" ]
  }, {
    "name" : "Install zeppelin",
    "uri" : "s3a://XXX/zeppelin_install.sh",
    "parameters" : "",
    "nodes" : [ "master" ],
    "active_master" : "true",
    "before_component_start" : "false",
    "start_time" : "1667892101",
    "state" : "IN_PROGRESS",
    "fail_action" : "continue",
    "action_stages" : [ "AFTER_SCALE_IN", "AFTER_SCALE_OUT" ]
  } ]
}
  • Create a cluster with the HA function disabled without using the node_groups parameter group. The cluster version is MRS 3.2.0-LTS.1. 
    POST https://{endpoint}/v1.1/{project_id}/run-job-flow

{
  "billing_type" : 12,
  "data_center" : "",
  "master_node_num" : 1,
  "master_node_size" : "s3.2xlarge.2.linux.bigdata",
  "core_node_num" : 1,
  "core_node_size" : "s3.2xlarge.2.linux.bigdata", 
  "available_zone_id" : "0e7a368b6c54493e94ad32666b47e23e", 
  "cluster_name" : "newcluster",
  "vpc" : "vpc1",
  "vpc_id" : "5b7db34d-3534-4a6e-ac94-023cd36aaf74",
  "subnet_id" : "815bece0-fd22-4b65-8a6e-15788c99ee43",
  "subnet_name" : "subnet",
  "security_groups_id" : "",
  "enterprise_project_id" : "0",
  "tags" : [ {
    "key" : "key1",
    "value" : "value1"
  }, {
    "key" : "key2",
    "value" : "value2"
  } ],
  "cluster_version" : "MRS 3.2.0-LTS.1",
  "cluster_type" : 0,
  "master_data_volume_type" : "SATA",
  "master_data_volume_size" : 600,
  "master_data_volume_count" : 1,
  "core_data_volume_type" : "SATA",
  "core_data_volume_size" : 600,
  "core_data_volume_count" : 1,
  "login_mode" : 1,
  "node_public_cert_name" : "SSHkey-bba1",
  "safe_mode" : 0,
  "cluster_admin_secret" : "******",
  "log_collection" : 1,
  "component_list" : [ {
    "component_name" : "Hadoop"
  }, {
    "component_name" : "Spark2x"
  }, {
    "component_name" : "HBase"
  }, {
    "component_name" : "Hive"
  }, {
    "component_name" : "Zookeeper"
  }, {
    "component_name" : "Tez"
  }, {
    "component_name" : "Hue"
  }, {
    "component_name" : "Loader"
  }, {
    "component_name" : "Flink"
  } ],
  "add_jobs" : [ {
    "job_type" : 1,
    "job_name" : "tenji111",
    "jar_path" : "s3a://bigdata/program/hadoop-mapreduce-examples-XXX.jar",
    "arguments" : "wordcount",
    "input" : "s3a://bigdata/input/wd_1k/",
    "output" : "s3a://bigdata/output/",
    "job_log" : "s3a://bigdata/log/",
    "shutdown_cluster" : false,
    "file_action" : "",
    "submit_job_once_cluster_run" : true,
    "hql" : "",
    "hive_script_path" : ""
  } ],
  "bootstrap_scripts" : [ {
    "name" : "Install zeppelin",
    "uri" : "s3a://XXX/zeppelin_install.sh",
    "parameters" : "",
    "nodes" : [ "master" ],
    "active_master" : "false",
    "before_component_start" : "false",
    "start_time" : "1667892101",
    "state" : "IN_PROGRESS",
    "fail_action" : "continue",
    "action_stages" : [ "AFTER_SCALE_IN", "AFTER_SCALE_OUT" ]
  } ]
}

Example Response

Status code: 200

The cluster is created.

{
  "cluster_id" : "da1592c2-bb7e-468d-9ac9-83246e95447a",
  "result" : true,
  "msg" : ""
}

SDK Sample Code

The SDK sample code is as follows.

  • Use the node_groups parameter group to create a cluster with the cluster HA feature enabled, with the cluster version of MRS 3.1.0.

      1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
    		 
    package com.huaweicloud.sdk.test;

import com.huaweicloud.sdk.core.auth.ICredential;
import com.huaweicloud.sdk.core.auth.BasicCredentials;
import com.huaweicloud.sdk.core.exception.ConnectionException;
import com.huaweicloud.sdk.core.exception.RequestTimeoutException;
import com.huaweicloud.sdk.core.exception.ServiceResponseException;
import com.huaweicloud.sdk.mrs.v1.region.MrsRegion;
import com.huaweicloud.sdk.mrs.v1.*;
import com.huaweicloud.sdk.mrs.v1.model.*;

import java.util.List;
import java.util.ArrayList;

public class CreateClusterSolution {

    public static void main(String[] args) {
        // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
        // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
        String ak = System.getenv("CLOUD_SDK_AK");
        String sk = System.getenv("CLOUD_SDK_SK");
        String projectId = "{project_id}";

        ICredential auth = new BasicCredentials()
                .withProjectId(projectId)
                .withAk(ak)
                .withSk(sk);

        MrsClient client = MrsClient.newBuilder()
                .withCredential(auth)
                .withRegion(MrsRegion.valueOf("<YOUR REGION>"))
                .build();
        CreateClusterRequest request = new CreateClusterRequest();
        CreateClusterReqV11 body = new CreateClusterReqV11();
        List<String> listExecScriptsNodes = new ArrayList<>();
        listExecScriptsNodes.add("master");
        listExecScriptsNodes.add("core");
        listExecScriptsNodes.add("task");
        List<String> listExecScriptsNodes1 = new ArrayList<>();
        listExecScriptsNodes1.add("master");
        listExecScriptsNodes1.add("core");
        listExecScriptsNodes1.add("task");
        List<ScaleScript> listAutoScalingPolicyExecScripts = new ArrayList<>();
        listAutoScalingPolicyExecScripts.add(
            new ScaleScript()
                .withName("before_scale_out")
                .withUri("s3a://XXX/zeppelin_install.sh")
                .withParameters("${mrs_scale_node_num} ${mrs_scale_type} xxx")
                .withNodes(listExecScriptsNodes1)
                .withActiveMaster(true)
                .withFailAction(ScaleScript.FailActionEnum.fromValue("continue"))
                .withActionStage(ScaleScript.ActionStageEnum.fromValue("before_scale_out"))
        );
        listAutoScalingPolicyExecScripts.add(
            new ScaleScript()
                .withName("after_scale_out")
                .withUri("s3a://XXX/storm_rebalance.sh")
                .withParameters("${mrs_scale_node_hostnames} ${mrs_scale_node_ips}")
                .withNodes(listExecScriptsNodes)
                .withActiveMaster(true)
                .withFailAction(ScaleScript.FailActionEnum.fromValue("continue"))
                .withActionStage(ScaleScript.ActionStageEnum.fromValue("after_scale_out"))
        );
        Trigger triggerRules = new Trigger();
        triggerRules.withMetricName("YARNMemoryAvailablePercentage")
            .withMetricValue("70")
            .withComparisonOperator("GT")
            .withEvaluationPeriods(10);
        Trigger triggerRules1 = new Trigger();
        triggerRules1.withMetricName("YARNMemoryAvailablePercentage")
            .withMetricValue("25")
            .withComparisonOperator("LT")
            .withEvaluationPeriods(10);
        List<Rule> listAutoScalingPolicyRules = new ArrayList<>();
        listAutoScalingPolicyRules.add(
            new Rule()
                .withName("default-expand-1")
                .withAdjustmentType(Rule.AdjustmentTypeEnum.fromValue("scale_out"))
                .withCoolDownMinutes(5)
                .withScalingAdjustment(1)
                .withTrigger(triggerRules1)
        );
        listAutoScalingPolicyRules.add(
            new Rule()
                .withName("default-shrink-1")
                .withAdjustmentType(Rule.AdjustmentTypeEnum.fromValue("scale_in"))
                .withCoolDownMinutes(5)
                .withScalingAdjustment(1)
                .withTrigger(triggerRules)
        );
        List<ResourcesPlan> listAutoScalingPolicyResourcesPlans = new ArrayList<>();
        listAutoScalingPolicyResourcesPlans.add(
            new ResourcesPlan()
                .withPeriodType("daily")
                .withStartTime("9:50")
                .withEndTime("10:20")
                .withMinCapacity(2)
                .withMaxCapacity(3)
        );
        listAutoScalingPolicyResourcesPlans.add(
            new ResourcesPlan()
                .withPeriodType("daily")
                .withStartTime("10:20")
                .withEndTime("12:30")
                .withMinCapacity(0)
                .withMaxCapacity(2)
        );
        AutoScalingPolicy autoScalingPolicyNodeGroups = new AutoScalingPolicy();
        autoScalingPolicyNodeGroups.withAutoScalingEnable(true)
            .withMinCapacity(1)
            .withMaxCapacity(3)
            .withResourcesPlans(listAutoScalingPolicyResourcesPlans)
            .withRules(listAutoScalingPolicyRules)
            .withExecScripts(listAutoScalingPolicyExecScripts);
        List<NodeGroupV11> listbodyNodeGroups = new ArrayList<>();
        listbodyNodeGroups.add(
            new NodeGroupV11()
                .withGroupName("master_node_default_group")
                .withNodeNum(2)
                .withNodeSize("s3.xlarge.2.linux.bigdata")
                .withRootVolumeSize("480")
                .withRootVolumeType("SATA")
                .withDataVolumeType("SATA")
                .withDataVolumeCount(1)
                .withDataVolumeSize(600)
        );
        listbodyNodeGroups.add(
            new NodeGroupV11()
                .withGroupName("core_node_analysis_group")
                .withNodeNum(3)
                .withNodeSize("s3.xlarge.2.linux.bigdata")
                .withRootVolumeSize("480")
                .withRootVolumeType("SATA")
                .withDataVolumeType("SATA")
                .withDataVolumeCount(1)
                .withDataVolumeSize(600)
        );
        listbodyNodeGroups.add(
            new NodeGroupV11()
                .withGroupName("task_node_analysis_group")
                .withNodeNum(2)
                .withNodeSize("s3.xlarge.2.linux.bigdata")
                .withRootVolumeSize("480")
                .withRootVolumeType("SATA")
                .withDataVolumeType("SATA")
                .withDataVolumeCount(0)
                .withDataVolumeSize(600)
                .withAutoScalingPolicy(autoScalingPolicyNodeGroups)
        );
        List<Tag> listbodyTags = new ArrayList<>();
        listbodyTags.add(
            new Tag()
                .withKey("key1")
                .withValue("value1")
        );
        listbodyTags.add(
            new Tag()
                .withKey("key2")
                .withValue("value2")
        );
        List<BootstrapScript.ActionStagesEnum> listBootstrapScriptsActionStages = new ArrayList<>();
        listBootstrapScriptsActionStages.add(BootstrapScript.ActionStagesEnum.fromValue("AFTER_SCALE_IN"));
        listBootstrapScriptsActionStages.add(BootstrapScript.ActionStagesEnum.fromValue("AFTER_SCALE_OUT"));
        List<String> listBootstrapScriptsNodes = new ArrayList<>();
        listBootstrapScriptsNodes.add("master");
        List<BootstrapScript.ActionStagesEnum> listBootstrapScriptsActionStages1 = new ArrayList<>();
        listBootstrapScriptsActionStages1.add(BootstrapScript.ActionStagesEnum.fromValue("BEFORE_COMPONENT_FIRST_START"));
        listBootstrapScriptsActionStages1.add(BootstrapScript.ActionStagesEnum.fromValue("BEFORE_SCALE_IN"));
        List<String> listBootstrapScriptsNodes1 = new ArrayList<>();
        listBootstrapScriptsNodes1.add("master");
        listBootstrapScriptsNodes1.add("core");
        listBootstrapScriptsNodes1.add("task");
        List<BootstrapScript> listbodyBootstrapScripts = new ArrayList<>();
        listbodyBootstrapScripts.add(
            new BootstrapScript()
                .withName("Modify os config")
                .withUri("s3a://XXX/modify_os_config.sh")
                .withParameters("param1 param2")
                .withNodes(listBootstrapScriptsNodes1)
                .withActiveMaster(false)
                .withFailAction(BootstrapScript.FailActionEnum.fromValue("continue"))
                .withBeforeComponentStart(true)
                .withStartTime(1667892101L)
                .withState(BootstrapScript.StateEnum.fromValue("IN_PROGRESS"))
                .withActionStages(listBootstrapScriptsActionStages1)
        );
        listbodyBootstrapScripts.add(
            new BootstrapScript()
                .withName("Install zepplin")
                .withUri("s3a://XXX/zeppelin_install.sh")
                .withParameters("")
                .withNodes(listBootstrapScriptsNodes)
                .withActiveMaster(true)
                .withFailAction(BootstrapScript.FailActionEnum.fromValue("continue"))
                .withBeforeComponentStart(false)
                .withStartTime(1667892101L)
                .withState(BootstrapScript.StateEnum.fromValue("IN_PROGRESS"))
                .withActionStages(listBootstrapScriptsActionStages)
        );
        List<AddJobsReqV11> listbodyAddJobs = new ArrayList<>();
        listbodyAddJobs.add(
            new AddJobsReqV11()
                .withJobType(1)
                .withJobName("tenji111")
                .withJarPath("s3a://bigdata/program/hadoop-mapreduce-examples-2.7.2.jar")
                .withArguments("wordcount")
                .withInput("s3a://bigdata/input/wd_1k/")
                .withOutput("s3a://bigdata/ouput/")
                .withJobLog("s3a://bigdata/log/")
                .withHiveScriptPath("")
                .withHql("")
                .withShutdownCluster(true)
                .withSubmitJobOnceClusterRun(true)
                .withFileAction("")
        );
        List<ComponentAmbV11> listbodyComponentList = new ArrayList<>();
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Hadoop")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Spark")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("HBase")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Hive")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Presto")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Tez")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Hue")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Loader")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Flink")
        );
        body.withNodeGroups(listbodyNodeGroups);
        body.withLoginMode(CreateClusterReqV11.LoginModeEnum.NUMBER_1);
        body.withTags(listbodyTags);
        body.withEnterpriseProjectId("0");
        body.withLogCollection(CreateClusterReqV11.LogCollectionEnum.NUMBER_1);
        body.withClusterType(CreateClusterReqV11.ClusterTypeEnum.NUMBER_0);
        body.withSafeMode(CreateClusterReqV11.SafeModeEnum.NUMBER_0);
        body.withClusterMasterSecret("");
        body.withClusterAdminSecret("");
        body.withBootstrapScripts(listbodyBootstrapScripts);
        body.withAddJobs(listbodyAddJobs);
        body.withSecurityGroupsId("4820eace-66ad-4f2c-8d46-cf340e3029dd");
        body.withSubnetName("subnet-4b44");
        body.withSubnetId("67984709-e15e-4e86-9886-d76712d4e00a");
        body.withVpcId("4a365717-67be-4f33-80c5-98e98a813af8");
        body.withAvailableZoneId("d573142f24894ef3bd3664de068b44b0");
        body.withComponentList(listbodyComponentList);
        body.withVpc("vpc-4b1c");
        body.withDataCenter("");
        body.withBillingType(CreateClusterReqV11.BillingTypeEnum.NUMBER_12);
        body.withClusterName("mrs_HEbK");
        body.withClusterVersion("MRS 3.1.0");
        request.withBody(body);
        try {
            CreateClusterResponse response = client.createCluster(request);
            System.out.println(response.toString());
        } catch (ConnectionException e) {
            e.printStackTrace();
        } catch (RequestTimeoutException e) {
            e.printStackTrace();
        } catch (ServiceResponseException e) {
            e.printStackTrace();
            System.out.println(e.getHttpStatusCode());
            System.out.println(e.getRequestId());
            System.out.println(e.getErrorCode());
            System.out.println(e.getErrorMsg());
        }
    }
}

  • Do not use the node_groups parameter group to create a cluster with the cluster HA feature enabled, with the cluster version of MRS 3.1.0.

      1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
    		 
    package com.huaweicloud.sdk.test;

import com.huaweicloud.sdk.core.auth.ICredential;
import com.huaweicloud.sdk.core.auth.BasicCredentials;
import com.huaweicloud.sdk.core.exception.ConnectionException;
import com.huaweicloud.sdk.core.exception.RequestTimeoutException;
import com.huaweicloud.sdk.core.exception.ServiceResponseException;
import com.huaweicloud.sdk.mrs.v1.region.MrsRegion;
import com.huaweicloud.sdk.mrs.v1.*;
import com.huaweicloud.sdk.mrs.v1.model.*;

import java.util.List;
import java.util.ArrayList;

public class CreateClusterSolution {

    public static void main(String[] args) {
        // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
        // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
        String ak = System.getenv("CLOUD_SDK_AK");
        String sk = System.getenv("CLOUD_SDK_SK");
        String projectId = "{project_id}";

        ICredential auth = new BasicCredentials()
                .withProjectId(projectId)
                .withAk(ak)
                .withSk(sk);

        MrsClient client = MrsClient.newBuilder()
                .withCredential(auth)
                .withRegion(MrsRegion.valueOf("<YOUR REGION>"))
                .build();
        CreateClusterRequest request = new CreateClusterRequest();
        CreateClusterReqV11 body = new CreateClusterReqV11();
        List<Tag> listbodyTags = new ArrayList<>();
        listbodyTags.add(
            new Tag()
                .withKey("key1")
                .withValue("value1")
        );
        listbodyTags.add(
            new Tag()
                .withKey("key2")
                .withValue("value2")
        );
        List<BootstrapScript.ActionStagesEnum> listBootstrapScriptsActionStages = new ArrayList<>();
        listBootstrapScriptsActionStages.add(BootstrapScript.ActionStagesEnum.fromValue("AFTER_SCALE_IN"));
        listBootstrapScriptsActionStages.add(BootstrapScript.ActionStagesEnum.fromValue("AFTER_SCALE_OUT"));
        List<String> listBootstrapScriptsNodes = new ArrayList<>();
        listBootstrapScriptsNodes.add("master");
        List<BootstrapScript.ActionStagesEnum> listBootstrapScriptsActionStages1 = new ArrayList<>();
        listBootstrapScriptsActionStages1.add(BootstrapScript.ActionStagesEnum.fromValue("BEFORE_COMPONENT_FIRST_START"));
        listBootstrapScriptsActionStages1.add(BootstrapScript.ActionStagesEnum.fromValue("BEFORE_SCALE_IN"));
        List<String> listBootstrapScriptsNodes1 = new ArrayList<>();
        listBootstrapScriptsNodes1.add("master");
        listBootstrapScriptsNodes1.add("core");
        listBootstrapScriptsNodes1.add("task");
        List<BootstrapScript> listbodyBootstrapScripts = new ArrayList<>();
        listbodyBootstrapScripts.add(
            new BootstrapScript()
                .withName("Modifyosconfig")
                .withUri("s3a://XXX/modify_os_config.sh")
                .withParameters("param1param2")
                .withNodes(listBootstrapScriptsNodes1)
                .withActiveMaster(false)
                .withFailAction(BootstrapScript.FailActionEnum.fromValue("continue"))
                .withBeforeComponentStart(true)
                .withStartTime(1667892101L)
                .withState(BootstrapScript.StateEnum.fromValue("IN_PROGRESS"))
                .withActionStages(listBootstrapScriptsActionStages1)
        );
        listbodyBootstrapScripts.add(
            new BootstrapScript()
                .withName("Installzepplin")
                .withUri("s3a://XXX/zeppelin_install.sh")
                .withParameters("")
                .withNodes(listBootstrapScriptsNodes)
                .withActiveMaster(true)
                .withFailAction(BootstrapScript.FailActionEnum.fromValue("continue"))
                .withBeforeComponentStart(false)
                .withStartTime(1667892101L)
                .withState(BootstrapScript.StateEnum.fromValue("IN_PROGRESS"))
                .withActionStages(listBootstrapScriptsActionStages)
        );
        List<String> listExecScriptsNodes = new ArrayList<>();
        listExecScriptsNodes.add("master");
        listExecScriptsNodes.add("core");
        listExecScriptsNodes.add("task");
        List<String> listExecScriptsNodes1 = new ArrayList<>();
        listExecScriptsNodes1.add("master");
        listExecScriptsNodes1.add("core");
        listExecScriptsNodes1.add("task");
        List<ScaleScript> listAutoScalingPolicyExecScripts = new ArrayList<>();
        listAutoScalingPolicyExecScripts.add(
            new ScaleScript()
                .withName("before_scale_out")
                .withUri("s3a://XXX/zeppelin_install.sh")
                .withParameters("${mrs_scale_node_num} ${mrs_scale_type}xxx")
                .withNodes(listExecScriptsNodes1)
                .withActiveMaster(true)
                .withFailAction(ScaleScript.FailActionEnum.fromValue("continue"))
                .withActionStage(ScaleScript.ActionStageEnum.fromValue("before_scale_out"))
        );
        listAutoScalingPolicyExecScripts.add(
            new ScaleScript()
                .withName("after_scale_out")
                .withUri("s3a://XXX/storm_rebalance.sh")
                .withParameters("${mrs_scale_node_hostnames} ${mrs_scale_node_ips}")
                .withNodes(listExecScriptsNodes)
                .withActiveMaster(true)
                .withFailAction(ScaleScript.FailActionEnum.fromValue("continue"))
                .withActionStage(ScaleScript.ActionStageEnum.fromValue("after_scale_out"))
        );
        Trigger triggerRules = new Trigger();
        triggerRules.withMetricName("YARNMemoryAvailablePercentage")
            .withMetricValue("70")
            .withComparisonOperator("GT")
            .withEvaluationPeriods(10);
        Trigger triggerRules1 = new Trigger();
        triggerRules1.withMetricName("YARNMemoryAvailablePercentage")
            .withMetricValue("25")
            .withComparisonOperator("LT")
            .withEvaluationPeriods(10);
        List<Rule> listAutoScalingPolicyRules = new ArrayList<>();
        listAutoScalingPolicyRules.add(
            new Rule()
                .withName("default-expand-1")
                .withAdjustmentType(Rule.AdjustmentTypeEnum.fromValue("scale_out"))
                .withCoolDownMinutes(5)
                .withScalingAdjustment(1)
                .withTrigger(triggerRules1)
        );
        listAutoScalingPolicyRules.add(
            new Rule()
                .withName("default-shrink-1")
                .withAdjustmentType(Rule.AdjustmentTypeEnum.fromValue("scale_in"))
                .withCoolDownMinutes(5)
                .withScalingAdjustment(1)
                .withTrigger(triggerRules)
        );
        List<ResourcesPlan> listAutoScalingPolicyResourcesPlans = new ArrayList<>();
        listAutoScalingPolicyResourcesPlans.add(
            new ResourcesPlan()
                .withPeriodType("daily")
                .withStartTime("9:50")
                .withEndTime("10:20")
                .withMinCapacity(2)
                .withMaxCapacity(3)
        );
        listAutoScalingPolicyResourcesPlans.add(
            new ResourcesPlan()
                .withPeriodType("daily")
                .withStartTime("10:20")
                .withEndTime("12:30")
                .withMinCapacity(0)
                .withMaxCapacity(2)
        );
        AutoScalingPolicy autoScalingPolicyTaskNodeGroups = new AutoScalingPolicy();
        autoScalingPolicyTaskNodeGroups.withAutoScalingEnable(true)
            .withMinCapacity(1)
            .withMaxCapacity(3)
            .withResourcesPlans(listAutoScalingPolicyResourcesPlans)
            .withRules(listAutoScalingPolicyRules)
            .withExecScripts(listAutoScalingPolicyExecScripts);
        List<TaskNodeGroup> listbodyTaskNodeGroups = new ArrayList<>();
        listbodyTaskNodeGroups.add(
            new TaskNodeGroup()
                .withNodeNum(2)
                .withNodeSize("s3.xlarge.2.linux.bigdata")
                .withDataVolumeType(TaskNodeGroup.DataVolumeTypeEnum.fromValue("SATA"))
                .withDataVolumeCount(1)
                .withDataVolumeSize(600)
                .withAutoScalingPolicy(autoScalingPolicyTaskNodeGroups)
        );
        List<AddJobsReqV11> listbodyAddJobs = new ArrayList<>();
        listbodyAddJobs.add(
            new AddJobsReqV11()
                .withJobType(1)
                .withJobName("tenji111")
                .withJarPath("s3a://bigdata/program/hadoop-mapreduce-examples-2.7.2.jar")
                .withArguments("wordcount")
                .withInput("s3a://bigdata/input/wd_1k/")
                .withOutput("s3a://bigdata/ouput/")
                .withJobLog("s3a://bigdata/log/")
                .withHiveScriptPath("")
                .withHql("")
                .withShutdownCluster(true)
                .withSubmitJobOnceClusterRun(true)
                .withFileAction("")
        );
        List<ComponentAmbV11> listbodyComponentList = new ArrayList<>();
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Hadoop")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Spark")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("HBase")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Hive")
        );
        body.withTags(listbodyTags);
        body.withLogCollection(CreateClusterReqV11.LogCollectionEnum.NUMBER_1);
        body.withClusterType(CreateClusterReqV11.ClusterTypeEnum.NUMBER_0);
        body.withSafeMode(CreateClusterReqV11.SafeModeEnum.NUMBER_0);
        body.withNodePublicCertName("SSHkey-bba1");
        body.withBootstrapScripts(listbodyBootstrapScripts);
        body.withTaskNodeGroups(listbodyTaskNodeGroups);
        body.withCoreDataVolumeCount(2);
        body.withCoreDataVolumeSize(600);
        body.withCoreDataVolumeType(CreateClusterReqV11.CoreDataVolumeTypeEnum.fromValue("SATA"));
        body.withMasterDataVolumeCount(CreateClusterReqV11.MasterDataVolumeCountEnum.NUMBER_1);
        body.withMasterDataVolumeSize(600);
        body.withMasterDataVolumeType(CreateClusterReqV11.MasterDataVolumeTypeEnum.fromValue("SATA"));
        body.withAddJobs(listbodyAddJobs);
        body.withSecurityGroupsId("845bece1-fd22-4b45-7a6e-14338c99ee43");
        body.withSubnetName("subnet");
        body.withSubnetId("815bece0-fd22-4b65-8a6e-15788c99ee43");
        body.withVpcId("5b7db34d-3534-4a6e-ac94-023cd36aaf74");
        body.withAvailableZoneId("d573142f24894ef3bd3664de068b44b0");
        body.withComponentList(listbodyComponentList);
        body.withCoreNodeSize("s1.xlarge.linux.bigdata");
        body.withMasterNodeSize("s3.2xlarge.2.linux.bigdata");
        body.withVpc("vpc1");
        body.withDataCenter("");
        body.withBillingType(CreateClusterReqV11.BillingTypeEnum.NUMBER_12);
        body.withCoreNodeNum(3);
        body.withMasterNodeNum(2);
        body.withClusterName("newcluster");
        body.withClusterVersion("MRS 3.1.0");
        request.withBody(body);
        try {
            CreateClusterResponse response = client.createCluster(request);
            System.out.println(response.toString());
        } catch (ConnectionException e) {
            e.printStackTrace();
        } catch (RequestTimeoutException e) {
            e.printStackTrace();
        } catch (ServiceResponseException e) {
            e.printStackTrace();
            System.out.println(e.getHttpStatusCode());
            System.out.println(e.getRequestId());
            System.out.println(e.getErrorCode());
            System.out.println(e.getErrorMsg());
        }
    }
}

  • Use the node_groups parameter group to create a cluster with the cluster HA feature disabled and the minimum specifications, with the cluster version of MRS 3.1.0.

      1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
    		 
    package com.huaweicloud.sdk.test;

import com.huaweicloud.sdk.core.auth.ICredential;
import com.huaweicloud.sdk.core.auth.BasicCredentials;
import com.huaweicloud.sdk.core.exception.ConnectionException;
import com.huaweicloud.sdk.core.exception.RequestTimeoutException;
import com.huaweicloud.sdk.core.exception.ServiceResponseException;
import com.huaweicloud.sdk.mrs.v1.region.MrsRegion;
import com.huaweicloud.sdk.mrs.v1.*;
import com.huaweicloud.sdk.mrs.v1.model.*;

import java.util.List;
import java.util.ArrayList;

public class CreateClusterSolution {

    public static void main(String[] args) {
        // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
        // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
        String ak = System.getenv("CLOUD_SDK_AK");
        String sk = System.getenv("CLOUD_SDK_SK");
        String projectId = "{project_id}";

        ICredential auth = new BasicCredentials()
                .withProjectId(projectId)
                .withAk(ak)
                .withSk(sk);

        MrsClient client = MrsClient.newBuilder()
                .withCredential(auth)
                .withRegion(MrsRegion.valueOf("<YOUR REGION>"))
                .build();
        CreateClusterRequest request = new CreateClusterRequest();
        CreateClusterReqV11 body = new CreateClusterReqV11();
        List<NodeGroupV11> listbodyNodeGroups = new ArrayList<>();
        listbodyNodeGroups.add(
            new NodeGroupV11()
                .withGroupName("master_node_default_group")
                .withNodeNum(1)
                .withNodeSize("s3.xlarge.2.linux.bigdata")
                .withRootVolumeSize("480")
                .withRootVolumeType("SATA")
                .withDataVolumeType("SATA")
                .withDataVolumeCount(1)
                .withDataVolumeSize(600)
        );
        listbodyNodeGroups.add(
            new NodeGroupV11()
                .withGroupName("core_node_analysis_group")
                .withNodeNum(1)
                .withNodeSize("s3.xlarge.2.linux.bigdata")
                .withRootVolumeSize("480")
                .withRootVolumeType("SATA")
                .withDataVolumeType("SATA")
                .withDataVolumeCount(1)
                .withDataVolumeSize(600)
        );
        List<Tag> listbodyTags = new ArrayList<>();
        listbodyTags.add(
            new Tag()
                .withKey("key1")
                .withValue("value1")
        );
        listbodyTags.add(
            new Tag()
                .withKey("key2")
                .withValue("value2")
        );
        List<BootstrapScript.ActionStagesEnum> listBootstrapScriptsActionStages = new ArrayList<>();
        listBootstrapScriptsActionStages.add(BootstrapScript.ActionStagesEnum.fromValue("AFTER_SCALE_IN"));
        listBootstrapScriptsActionStages.add(BootstrapScript.ActionStagesEnum.fromValue("AFTER_SCALE_OUT"));
        List<String> listBootstrapScriptsNodes = new ArrayList<>();
        listBootstrapScriptsNodes.add("master");
        List<BootstrapScript.ActionStagesEnum> listBootstrapScriptsActionStages1 = new ArrayList<>();
        listBootstrapScriptsActionStages1.add(BootstrapScript.ActionStagesEnum.fromValue("BEFORE_COMPONENT_FIRST_START"));
        listBootstrapScriptsActionStages1.add(BootstrapScript.ActionStagesEnum.fromValue("BEFORE_SCALE_IN"));
        List<String> listBootstrapScriptsNodes1 = new ArrayList<>();
        listBootstrapScriptsNodes1.add("master");
        listBootstrapScriptsNodes1.add("core");
        listBootstrapScriptsNodes1.add("task");
        List<BootstrapScript> listbodyBootstrapScripts = new ArrayList<>();
        listbodyBootstrapScripts.add(
            new BootstrapScript()
                .withName("Modify os config")
                .withUri("s3a://XXX/modify_os_config.sh")
                .withParameters("param1 param2")
                .withNodes(listBootstrapScriptsNodes1)
                .withActiveMaster(false)
                .withFailAction(BootstrapScript.FailActionEnum.fromValue("continue"))
                .withBeforeComponentStart(true)
                .withStartTime(1667892101L)
                .withState(BootstrapScript.StateEnum.fromValue("IN_PROGRESS"))
                .withActionStages(listBootstrapScriptsActionStages1)
        );
        listbodyBootstrapScripts.add(
            new BootstrapScript()
                .withName("Install zepplin")
                .withUri("s3a://XXX/zeppelin_install.sh")
                .withParameters("")
                .withNodes(listBootstrapScriptsNodes)
                .withActiveMaster(true)
                .withFailAction(BootstrapScript.FailActionEnum.fromValue("continue"))
                .withBeforeComponentStart(false)
                .withStartTime(1667892101L)
                .withState(BootstrapScript.StateEnum.fromValue("IN_PROGRESS"))
                .withActionStages(listBootstrapScriptsActionStages)
        );
        List<AddJobsReqV11> listbodyAddJobs = new ArrayList<>();
        listbodyAddJobs.add(
            new AddJobsReqV11()
                .withJobType(1)
                .withJobName("tenji111")
                .withJarPath("s3a://bigdata/program/hadoop-mapreduce-examples-2.7.2.jar")
                .withArguments("wordcount")
                .withInput("s3a://bigdata/input/wd_1k/")
                .withOutput("s3a://bigdata/ouput/")
                .withJobLog("s3a://bigdata/log/")
                .withHiveScriptPath("")
                .withHql("")
                .withShutdownCluster(true)
                .withSubmitJobOnceClusterRun(true)
                .withFileAction("")
        );
        List<ComponentAmbV11> listbodyComponentList = new ArrayList<>();
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Hadoop")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Spark")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("HBase")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Hive")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Presto")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Tez")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Hue")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Loader")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Flink")
        );
        body.withNodeGroups(listbodyNodeGroups);
        body.withLoginMode(CreateClusterReqV11.LoginModeEnum.NUMBER_1);
        body.withTags(listbodyTags);
        body.withEnterpriseProjectId("0");
        body.withLogCollection(CreateClusterReqV11.LogCollectionEnum.NUMBER_1);
        body.withClusterType(CreateClusterReqV11.ClusterTypeEnum.NUMBER_0);
        body.withSafeMode(CreateClusterReqV11.SafeModeEnum.NUMBER_0);
        body.withClusterMasterSecret("");
        body.withClusterAdminSecret("");
        body.withBootstrapScripts(listbodyBootstrapScripts);
        body.withAddJobs(listbodyAddJobs);
        body.withSecurityGroupsId("4820eace-66ad-4f2c-8d46-cf340e3029dd");
        body.withSubnetName("subnet-4b44");
        body.withSubnetId("67984709-e15e-4e86-9886-d76712d4e00a");
        body.withVpcId("4a365717-67be-4f33-80c5-98e98a813af8");
        body.withAvailableZoneId("d573142f24894ef3bd3664de068b44b0");
        body.withComponentList(listbodyComponentList);
        body.withVpc("vpc-4b1c");
        body.withDataCenter("");
        body.withBillingType(CreateClusterReqV11.BillingTypeEnum.NUMBER_12);
        body.withClusterName("mrs_HEbK");
        body.withClusterVersion("MRS 3.1.0");
        request.withBody(body);
        try {
            CreateClusterResponse response = client.createCluster(request);
            System.out.println(response.toString());
        } catch (ConnectionException e) {
            e.printStackTrace();
        } catch (RequestTimeoutException e) {
            e.printStackTrace();
        } catch (ServiceResponseException e) {
            e.printStackTrace();
            System.out.println(e.getHttpStatusCode());
            System.out.println(e.getRequestId());
            System.out.println(e.getErrorCode());
            System.out.println(e.getErrorMsg());
        }
    }
}

  • Do not use the node_groups parameter group to create a cluster with the cluster HA feature disabled and the minimum specifications, with the cluster version of MRS 3.1.0.

      1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
    		 
    package com.huaweicloud.sdk.test;

import com.huaweicloud.sdk.core.auth.ICredential;
import com.huaweicloud.sdk.core.auth.BasicCredentials;
import com.huaweicloud.sdk.core.exception.ConnectionException;
import com.huaweicloud.sdk.core.exception.RequestTimeoutException;
import com.huaweicloud.sdk.core.exception.ServiceResponseException;
import com.huaweicloud.sdk.mrs.v1.region.MrsRegion;
import com.huaweicloud.sdk.mrs.v1.*;
import com.huaweicloud.sdk.mrs.v1.model.*;

import java.util.List;
import java.util.ArrayList;

public class CreateClusterSolution {

    public static void main(String[] args) {
        // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
        // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
        String ak = System.getenv("CLOUD_SDK_AK");
        String sk = System.getenv("CLOUD_SDK_SK");
        String projectId = "{project_id}";

        ICredential auth = new BasicCredentials()
                .withProjectId(projectId)
                .withAk(ak)
                .withSk(sk);

        MrsClient client = MrsClient.newBuilder()
                .withCredential(auth)
                .withRegion(MrsRegion.valueOf("<YOUR REGION>"))
                .build();
        CreateClusterRequest request = new CreateClusterRequest();
        CreateClusterReqV11 body = new CreateClusterReqV11();
        List<Tag> listbodyTags = new ArrayList<>();
        listbodyTags.add(
            new Tag()
                .withKey("key1")
                .withValue("value1")
        );
        listbodyTags.add(
            new Tag()
                .withKey("key2")
                .withValue("value2")
        );
        List<BootstrapScript.ActionStagesEnum> listBootstrapScriptsActionStages = new ArrayList<>();
        listBootstrapScriptsActionStages.add(BootstrapScript.ActionStagesEnum.fromValue("AFTER_SCALE_IN"));
        listBootstrapScriptsActionStages.add(BootstrapScript.ActionStagesEnum.fromValue("AFTER_SCALE_OUT"));
        List<String> listBootstrapScriptsNodes = new ArrayList<>();
        listBootstrapScriptsNodes.add("master");
        List<BootstrapScript> listbodyBootstrapScripts = new ArrayList<>();
        listbodyBootstrapScripts.add(
            new BootstrapScript()
                .withName("Install zepplin")
                .withUri("s3a://XXX/zeppelin_install.sh")
                .withParameters("")
                .withNodes(listBootstrapScriptsNodes)
                .withActiveMaster(false)
                .withFailAction(BootstrapScript.FailActionEnum.fromValue("continue"))
                .withBeforeComponentStart(false)
                .withStartTime(1667892101L)
                .withState(BootstrapScript.StateEnum.fromValue("IN_PROGRESS"))
                .withActionStages(listBootstrapScriptsActionStages)
        );
        List<AddJobsReqV11> listbodyAddJobs = new ArrayList<>();
        listbodyAddJobs.add(
            new AddJobsReqV11()
                .withJobType(1)
                .withJobName("tenji111")
                .withJarPath("s3a://bigdata/program/hadoop-mapreduce-examples-XXX.jar")
                .withArguments("wordcount")
                .withInput("s3a://bigdata/input/wd_1k/")
                .withOutput("s3a://bigdata/ouput/")
                .withJobLog("s3a://bigdata/log/")
                .withHiveScriptPath("")
                .withHql("")
                .withShutdownCluster(false)
                .withSubmitJobOnceClusterRun(true)
                .withFileAction("")
        );
        List<ComponentAmbV11> listbodyComponentList = new ArrayList<>();
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Hadoop")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Spark")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("HBase")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Hive")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Presto")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Tez")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Hue")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Loader")
        );
        listbodyComponentList.add(
            new ComponentAmbV11()
                .withComponentName("Flink")
        );
        body.withLoginMode(CreateClusterReqV11.LoginModeEnum.NUMBER_1);
        body.withTags(listbodyTags);
        body.withEnterpriseProjectId("0");
        body.withLogCollection(CreateClusterReqV11.LogCollectionEnum.NUMBER_1);
        body.withClusterType(CreateClusterReqV11.ClusterTypeEnum.NUMBER_0);
        body.withSafeMode(CreateClusterReqV11.SafeModeEnum.NUMBER_0);
        body.withClusterAdminSecret("******");
        body.withNodePublicCertName("SSHkey-bba1");
        body.withBootstrapScripts(listbodyBootstrapScripts);
        body.withCoreDataVolumeCount(1);
        body.withCoreDataVolumeSize(600);
        body.withCoreDataVolumeType(CreateClusterReqV11.CoreDataVolumeTypeEnum.fromValue("SATA"));
        body.withMasterDataVolumeCount(CreateClusterReqV11.MasterDataVolumeCountEnum.NUMBER_1);
        body.withMasterDataVolumeSize(600);
        body.withMasterDataVolumeType(CreateClusterReqV11.MasterDataVolumeTypeEnum.fromValue("SATA"));
        body.withAddJobs(listbodyAddJobs);
        body.withSecurityGroupsId("");
        body.withSubnetName("subnet");
        body.withSubnetId("815bece0-fd22-4b65-8a6e-15788c99ee43");
        body.withVpcId("5b7db34d-3534-4a6e-ac94-023cd36aaf74");
        body.withAvailableZoneId("d573142f24894ef3bd3664de068b44b0");
        body.withComponentList(listbodyComponentList);
        body.withCoreNodeSize("s1.xlarge.linux.bigdata");
        body.withMasterNodeSize("s3.2xlarge.2.linux.bigdata");
        body.withVpc("vpc1");
        body.withDataCenter("");
        body.withBillingType(CreateClusterReqV11.BillingTypeEnum.NUMBER_12);
        body.withCoreNodeNum(1);
        body.withMasterNodeNum(1);
        body.withClusterName("newcluster");
        body.withClusterVersion("MRS 3.1.0");
        request.withBody(body);
        try {
            CreateClusterResponse response = client.createCluster(request);
            System.out.println(response.toString());
        } catch (ConnectionException e) {
            e.printStackTrace();
        } catch (RequestTimeoutException e) {
            e.printStackTrace();
        } catch (ServiceResponseException e) {
            e.printStackTrace();
            System.out.println(e.getHttpStatusCode());
            System.out.println(e.getRequestId());
            System.out.println(e.getErrorCode());
            System.out.println(e.getErrorMsg());
        }
    }
}

  • Use the node_groups parameter group to create a cluster with the cluster HA feature enabled, with the cluster version of MRS 3.1.0.

      1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
    		 
    # coding: utf-8

import os
from huaweicloudsdkcore.auth.credentials import BasicCredentials
from huaweicloudsdkmrs.v1.region.mrs_region import MrsRegion
from huaweicloudsdkcore.exceptions import exceptions
from huaweicloudsdkmrs.v1 import *

if __name__ == "__main__":
    # The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
    # In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
    ak = os.environ["CLOUD_SDK_AK"]
    sk = os.environ["CLOUD_SDK_SK"]
    projectId = "{project_id}"

    credentials = BasicCredentials(ak, sk, projectId)

    client = MrsClient.new_builder() \
        .with_credentials(credentials) \
        .with_region(MrsRegion.value_of("<YOUR REGION>")) \
        .build()

    try:
        request = CreateClusterRequest()
        listNodesExecScripts = [
            "master",
            "core",
            "task"
        ]
        listNodesExecScripts1 = [
            "master",
            "core",
            "task"
        ]
        listExecScriptsAutoScalingPolicy = [
            ScaleScript(
                name="before_scale_out",
                uri="s3a://XXX/zeppelin_install.sh",
                parameters="${mrs_scale_node_num} ${mrs_scale_type} xxx",
                nodes=listNodesExecScripts1,
                active_master=True,
                fail_action="continue",
                action_stage="before_scale_out"
            ),
            ScaleScript(
                name="after_scale_out",
                uri="s3a://XXX/storm_rebalance.sh",
                parameters="${mrs_scale_node_hostnames} ${mrs_scale_node_ips}",
                nodes=listNodesExecScripts,
                active_master=True,
                fail_action="continue",
                action_stage="after_scale_out"
            )
        ]
        triggerRules = Trigger(
            metric_name="YARNMemoryAvailablePercentage",
            metric_value="70",
            comparison_operator="GT",
            evaluation_periods=10
        )
        triggerRules1 = Trigger(
            metric_name="YARNMemoryAvailablePercentage",
            metric_value="25",
            comparison_operator="LT",
            evaluation_periods=10
        )
        listRulesAutoScalingPolicy = [
            Rule(
                name="default-expand-1",
                adjustment_type="scale_out",
                cool_down_minutes=5,
                scaling_adjustment=1,
                trigger=triggerRules1
            ),
            Rule(
                name="default-shrink-1",
                adjustment_type="scale_in",
                cool_down_minutes=5,
                scaling_adjustment=1,
                trigger=triggerRules
            )
        ]
        listResourcesPlansAutoScalingPolicy = [
            ResourcesPlan(
                period_type="daily",
                start_time="9:50",
                end_time="10:20",
                min_capacity=2,
                max_capacity=3
            ),
            ResourcesPlan(
                period_type="daily",
                start_time="10:20",
                end_time="12:30",
                min_capacity=0,
                max_capacity=2
            )
        ]
        autoScalingPolicyNodeGroups = AutoScalingPolicy(
            auto_scaling_enable=True,
            min_capacity=1,
            max_capacity=3,
            resources_plans=listResourcesPlansAutoScalingPolicy,
            rules=listRulesAutoScalingPolicy,
            exec_scripts=listExecScriptsAutoScalingPolicy
        )
        listNodeGroupsbody = [
            NodeGroupV11(
                group_name="master_node_default_group",
                node_num=2,
                node_size="s3.xlarge.2.linux.bigdata",
                root_volume_size="480",
                root_volume_type="SATA",
                data_volume_type="SATA",
                data_volume_count=1,
                data_volume_size=600
            ),
            NodeGroupV11(
                group_name="core_node_analysis_group",
                node_num=3,
                node_size="s3.xlarge.2.linux.bigdata",
                root_volume_size="480",
                root_volume_type="SATA",
                data_volume_type="SATA",
                data_volume_count=1,
                data_volume_size=600
            ),
            NodeGroupV11(
                group_name="task_node_analysis_group",
                node_num=2,
                node_size="s3.xlarge.2.linux.bigdata",
                root_volume_size="480",
                root_volume_type="SATA",
                data_volume_type="SATA",
                data_volume_count=0,
                data_volume_size=600,
                auto_scaling_policy=autoScalingPolicyNodeGroups
            )
        ]
        listTagsbody = [
            Tag(
                key="key1",
                value="value1"
            ),
            Tag(
                key="key2",
                value="value2"
            )
        ]
        listActionStagesBootstrapScripts = [
            "AFTER_SCALE_IN",
            "AFTER_SCALE_OUT"
        ]
        listNodesBootstrapScripts = [
            "master"
        ]
        listActionStagesBootstrapScripts1 = [
            "BEFORE_COMPONENT_FIRST_START",
            "BEFORE_SCALE_IN"
        ]
        listNodesBootstrapScripts1 = [
            "master",
            "core",
            "task"
        ]
        listBootstrapScriptsbody = [
            BootstrapScript(
                name="Modify os config",
                uri="s3a://XXX/modify_os_config.sh",
                parameters="param1 param2",
                nodes=listNodesBootstrapScripts1,
                active_master=False,
                fail_action="continue",
                before_component_start=True,
                start_time=1667892101,
                state="IN_PROGRESS",
                action_stages=listActionStagesBootstrapScripts1
            ),
            BootstrapScript(
                name="Install zepplin",
                uri="s3a://XXX/zeppelin_install.sh",
                parameters="",
                nodes=listNodesBootstrapScripts,
                active_master=True,
                fail_action="continue",
                before_component_start=False,
                start_time=1667892101,
                state="IN_PROGRESS",
                action_stages=listActionStagesBootstrapScripts
            )
        ]
        listAddJobsbody = [
            AddJobsReqV11(
                job_type=1,
                job_name="tenji111",
                jar_path="s3a://bigdata/program/hadoop-mapreduce-examples-2.7.2.jar",
                arguments="wordcount",
                input="s3a://bigdata/input/wd_1k/",
                output="s3a://bigdata/ouput/",
                job_log="s3a://bigdata/log/",
                hive_script_path="",
                hql="",
                shutdown_cluster=True,
                submit_job_once_cluster_run=True,
                file_action=""
            )
        ]
        listComponentListbody = [
            ComponentAmbV11(
                component_name="Hadoop"
            ),
            ComponentAmbV11(
                component_name="Spark"
            ),
            ComponentAmbV11(
                component_name="HBase"
            ),
            ComponentAmbV11(
                component_name="Hive"
            ),
            ComponentAmbV11(
                component_name="Presto"
            ),
            ComponentAmbV11(
                component_name="Tez"
            ),
            ComponentAmbV11(
                component_name="Hue"
            ),
            ComponentAmbV11(
                component_name="Loader"
            ),
            ComponentAmbV11(
                component_name="Flink"
            )
        ]
        request.body = CreateClusterReqV11(
            node_groups=listNodeGroupsbody,
            login_mode=1,
            tags=listTagsbody,
            enterprise_project_id="0",
            log_collection=1,
            cluster_type=0,
            safe_mode=0,
            cluster_master_secret="",
            cluster_admin_secret="",
            bootstrap_scripts=listBootstrapScriptsbody,
            add_jobs=listAddJobsbody,
            security_groups_id="4820eace-66ad-4f2c-8d46-cf340e3029dd",
            subnet_name="subnet-4b44",
            subnet_id="67984709-e15e-4e86-9886-d76712d4e00a",
            vpc_id="4a365717-67be-4f33-80c5-98e98a813af8",
            available_zone_id="d573142f24894ef3bd3664de068b44b0",
            component_list=listComponentListbody,
            vpc="vpc-4b1c",
            data_center="",
            billing_type=12,
            cluster_name="mrs_HEbK",
            cluster_version="MRS 3.1.0"
        )
        response = client.create_cluster(request)
        print(response)
    except exceptions.ClientRequestException as e:
        print(e.status_code)
        print(e.request_id)
        print(e.error_code)
        print(e.error_msg)

  • Do not use the node_groups parameter group to create a cluster with the cluster HA feature enabled, with the cluster version of MRS 3.1.0.

      1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
    		 
    # coding: utf-8

import os
from huaweicloudsdkcore.auth.credentials import BasicCredentials
from huaweicloudsdkmrs.v1.region.mrs_region import MrsRegion
from huaweicloudsdkcore.exceptions import exceptions
from huaweicloudsdkmrs.v1 import *

if __name__ == "__main__":
    # The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
    # In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
    ak = os.environ["CLOUD_SDK_AK"]
    sk = os.environ["CLOUD_SDK_SK"]
    projectId = "{project_id}"

    credentials = BasicCredentials(ak, sk, projectId)

    client = MrsClient.new_builder() \
        .with_credentials(credentials) \
        .with_region(MrsRegion.value_of("<YOUR REGION>")) \
        .build()

    try:
        request = CreateClusterRequest()
        listTagsbody = [
            Tag(
                key="key1",
                value="value1"
            ),
            Tag(
                key="key2",
                value="value2"
            )
        ]
        listActionStagesBootstrapScripts = [
            "AFTER_SCALE_IN",
            "AFTER_SCALE_OUT"
        ]
        listNodesBootstrapScripts = [
            "master"
        ]
        listActionStagesBootstrapScripts1 = [
            "BEFORE_COMPONENT_FIRST_START",
            "BEFORE_SCALE_IN"
        ]
        listNodesBootstrapScripts1 = [
            "master",
            "core",
            "task"
        ]
        listBootstrapScriptsbody = [
            BootstrapScript(
                name="Modifyosconfig",
                uri="s3a://XXX/modify_os_config.sh",
                parameters="param1param2",
                nodes=listNodesBootstrapScripts1,
                active_master=False,
                fail_action="continue",
                before_component_start=True,
                start_time=1667892101,
                state="IN_PROGRESS",
                action_stages=listActionStagesBootstrapScripts1
            ),
            BootstrapScript(
                name="Installzepplin",
                uri="s3a://XXX/zeppelin_install.sh",
                parameters="",
                nodes=listNodesBootstrapScripts,
                active_master=True,
                fail_action="continue",
                before_component_start=False,
                start_time=1667892101,
                state="IN_PROGRESS",
                action_stages=listActionStagesBootstrapScripts
            )
        ]
        listNodesExecScripts = [
            "master",
            "core",
            "task"
        ]
        listNodesExecScripts1 = [
            "master",
            "core",
            "task"
        ]
        listExecScriptsAutoScalingPolicy = [
            ScaleScript(
                name="before_scale_out",
                uri="s3a://XXX/zeppelin_install.sh",
                parameters="${mrs_scale_node_num} ${mrs_scale_type}xxx",
                nodes=listNodesExecScripts1,
                active_master=True,
                fail_action="continue",
                action_stage="before_scale_out"
            ),
            ScaleScript(
                name="after_scale_out",
                uri="s3a://XXX/storm_rebalance.sh",
                parameters="${mrs_scale_node_hostnames} ${mrs_scale_node_ips}",
                nodes=listNodesExecScripts,
                active_master=True,
                fail_action="continue",
                action_stage="after_scale_out"
            )
        ]
        triggerRules = Trigger(
            metric_name="YARNMemoryAvailablePercentage",
            metric_value="70",
            comparison_operator="GT",
            evaluation_periods=10
        )
        triggerRules1 = Trigger(
            metric_name="YARNMemoryAvailablePercentage",
            metric_value="25",
            comparison_operator="LT",
            evaluation_periods=10
        )
        listRulesAutoScalingPolicy = [
            Rule(
                name="default-expand-1",
                adjustment_type="scale_out",
                cool_down_minutes=5,
                scaling_adjustment=1,
                trigger=triggerRules1
            ),
            Rule(
                name="default-shrink-1",
                adjustment_type="scale_in",
                cool_down_minutes=5,
                scaling_adjustment=1,
                trigger=triggerRules
            )
        ]
        listResourcesPlansAutoScalingPolicy = [
            ResourcesPlan(
                period_type="daily",
                start_time="9:50",
                end_time="10:20",
                min_capacity=2,
                max_capacity=3
            ),
            ResourcesPlan(
                period_type="daily",
                start_time="10:20",
                end_time="12:30",
                min_capacity=0,
                max_capacity=2
            )
        ]
        autoScalingPolicyTaskNodeGroups = AutoScalingPolicy(
            auto_scaling_enable=True,
            min_capacity=1,
            max_capacity=3,
            resources_plans=listResourcesPlansAutoScalingPolicy,
            rules=listRulesAutoScalingPolicy,
            exec_scripts=listExecScriptsAutoScalingPolicy
        )
        listTaskNodeGroupsbody = [
            TaskNodeGroup(
                node_num=2,
                node_size="s3.xlarge.2.linux.bigdata",
                data_volume_type="SATA",
                data_volume_count=1,
                data_volume_size=600,
                auto_scaling_policy=autoScalingPolicyTaskNodeGroups
            )
        ]
        listAddJobsbody = [
            AddJobsReqV11(
                job_type=1,
                job_name="tenji111",
                jar_path="s3a://bigdata/program/hadoop-mapreduce-examples-2.7.2.jar",
                arguments="wordcount",
                input="s3a://bigdata/input/wd_1k/",
                output="s3a://bigdata/ouput/",
                job_log="s3a://bigdata/log/",
                hive_script_path="",
                hql="",
                shutdown_cluster=True,
                submit_job_once_cluster_run=True,
                file_action=""
            )
        ]
        listComponentListbody = [
            ComponentAmbV11(
                component_name="Hadoop"
            ),
            ComponentAmbV11(
                component_name="Spark"
            ),
            ComponentAmbV11(
                component_name="HBase"
            ),
            ComponentAmbV11(
                component_name="Hive"
            )
        ]
        request.body = CreateClusterReqV11(
            tags=listTagsbody,
            log_collection=1,
            cluster_type=0,
            safe_mode=0,
            node_public_cert_name="SSHkey-bba1",
            bootstrap_scripts=listBootstrapScriptsbody,
            task_node_groups=listTaskNodeGroupsbody,
            core_data_volume_count=2,
            core_data_volume_size=600,
            core_data_volume_type="SATA",
            master_data_volume_count=1,
            master_data_volume_size=600,
            master_data_volume_type="SATA",
            add_jobs=listAddJobsbody,
            security_groups_id="845bece1-fd22-4b45-7a6e-14338c99ee43",
            subnet_name="subnet",
            subnet_id="815bece0-fd22-4b65-8a6e-15788c99ee43",
            vpc_id="5b7db34d-3534-4a6e-ac94-023cd36aaf74",
            available_zone_id="d573142f24894ef3bd3664de068b44b0",
            component_list=listComponentListbody,
            core_node_size="s1.xlarge.linux.bigdata",
            master_node_size="s3.2xlarge.2.linux.bigdata",
            vpc="vpc1",
            data_center="",
            billing_type=12,
            core_node_num=3,
            master_node_num=2,
            cluster_name="newcluster",
            cluster_version="MRS 3.1.0"
        )
        response = client.create_cluster(request)
        print(response)
    except exceptions.ClientRequestException as e:
        print(e.status_code)
        print(e.request_id)
        print(e.error_code)
        print(e.error_msg)

  • Use the node_groups parameter group to create a cluster with the cluster HA feature disabled and the minimum specifications, with the cluster version of MRS 3.1.0.

      1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
    		 
    # coding: utf-8

import os
from huaweicloudsdkcore.auth.credentials import BasicCredentials
from huaweicloudsdkmrs.v1.region.mrs_region import MrsRegion
from huaweicloudsdkcore.exceptions import exceptions
from huaweicloudsdkmrs.v1 import *

if __name__ == "__main__":
    # The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
    # In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
    ak = os.environ["CLOUD_SDK_AK"]
    sk = os.environ["CLOUD_SDK_SK"]
    projectId = "{project_id}"

    credentials = BasicCredentials(ak, sk, projectId)

    client = MrsClient.new_builder() \
        .with_credentials(credentials) \
        .with_region(MrsRegion.value_of("<YOUR REGION>")) \
        .build()

    try:
        request = CreateClusterRequest()
        listNodeGroupsbody = [
            NodeGroupV11(
                group_name="master_node_default_group",
                node_num=1,
                node_size="s3.xlarge.2.linux.bigdata",
                root_volume_size="480",
                root_volume_type="SATA",
                data_volume_type="SATA",
                data_volume_count=1,
                data_volume_size=600
            ),
            NodeGroupV11(
                group_name="core_node_analysis_group",
                node_num=1,
                node_size="s3.xlarge.2.linux.bigdata",
                root_volume_size="480",
                root_volume_type="SATA",
                data_volume_type="SATA",
                data_volume_count=1,
                data_volume_size=600
            )
        ]
        listTagsbody = [
            Tag(
                key="key1",
                value="value1"
            ),
            Tag(
                key="key2",
                value="value2"
            )
        ]
        listActionStagesBootstrapScripts = [
            "AFTER_SCALE_IN",
            "AFTER_SCALE_OUT"
        ]
        listNodesBootstrapScripts = [
            "master"
        ]
        listActionStagesBootstrapScripts1 = [
            "BEFORE_COMPONENT_FIRST_START",
            "BEFORE_SCALE_IN"
        ]
        listNodesBootstrapScripts1 = [
            "master",
            "core",
            "task"
        ]
        listBootstrapScriptsbody = [
            BootstrapScript(
                name="Modify os config",
                uri="s3a://XXX/modify_os_config.sh",
                parameters="param1 param2",
                nodes=listNodesBootstrapScripts1,
                active_master=False,
                fail_action="continue",
                before_component_start=True,
                start_time=1667892101,
                state="IN_PROGRESS",
                action_stages=listActionStagesBootstrapScripts1
            ),
            BootstrapScript(
                name="Install zepplin",
                uri="s3a://XXX/zeppelin_install.sh",
                parameters="",
                nodes=listNodesBootstrapScripts,
                active_master=True,
                fail_action="continue",
                before_component_start=False,
                start_time=1667892101,
                state="IN_PROGRESS",
                action_stages=listActionStagesBootstrapScripts
            )
        ]
        listAddJobsbody = [
            AddJobsReqV11(
                job_type=1,
                job_name="tenji111",
                jar_path="s3a://bigdata/program/hadoop-mapreduce-examples-2.7.2.jar",
                arguments="wordcount",
                input="s3a://bigdata/input/wd_1k/",
                output="s3a://bigdata/ouput/",
                job_log="s3a://bigdata/log/",
                hive_script_path="",
                hql="",
                shutdown_cluster=True,
                submit_job_once_cluster_run=True,
                file_action=""
            )
        ]
        listComponentListbody = [
            ComponentAmbV11(
                component_name="Hadoop"
            ),
            ComponentAmbV11(
                component_name="Spark"
            ),
            ComponentAmbV11(
                component_name="HBase"
            ),
            ComponentAmbV11(
                component_name="Hive"
            ),
            ComponentAmbV11(
                component_name="Presto"
            ),
            ComponentAmbV11(
                component_name="Tez"
            ),
            ComponentAmbV11(
                component_name="Hue"
            ),
            ComponentAmbV11(
                component_name="Loader"
            ),
            ComponentAmbV11(
                component_name="Flink"
            )
        ]
        request.body = CreateClusterReqV11(
            node_groups=listNodeGroupsbody,
            login_mode=1,
            tags=listTagsbody,
            enterprise_project_id="0",
            log_collection=1,
            cluster_type=0,
            safe_mode=0,
            cluster_master_secret="",
            cluster_admin_secret="",
            bootstrap_scripts=listBootstrapScriptsbody,
            add_jobs=listAddJobsbody,
            security_groups_id="4820eace-66ad-4f2c-8d46-cf340e3029dd",
            subnet_name="subnet-4b44",
            subnet_id="67984709-e15e-4e86-9886-d76712d4e00a",
            vpc_id="4a365717-67be-4f33-80c5-98e98a813af8",
            available_zone_id="d573142f24894ef3bd3664de068b44b0",
            component_list=listComponentListbody,
            vpc="vpc-4b1c",
            data_center="",
            billing_type=12,
            cluster_name="mrs_HEbK",
            cluster_version="MRS 3.1.0"
        )
        response = client.create_cluster(request)
        print(response)
    except exceptions.ClientRequestException as e:
        print(e.status_code)
        print(e.request_id)
        print(e.error_code)
        print(e.error_msg)

  • Do not use the node_groups parameter group to create a cluster with the cluster HA feature disabled and the minimum specifications, with the cluster version of MRS 3.1.0.

      1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
    		 
    # coding: utf-8

import os
from huaweicloudsdkcore.auth.credentials import BasicCredentials
from huaweicloudsdkmrs.v1.region.mrs_region import MrsRegion
from huaweicloudsdkcore.exceptions import exceptions
from huaweicloudsdkmrs.v1 import *

if __name__ == "__main__":
    # The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
    # In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
    ak = os.environ["CLOUD_SDK_AK"]
    sk = os.environ["CLOUD_SDK_SK"]
    projectId = "{project_id}"

    credentials = BasicCredentials(ak, sk, projectId)

    client = MrsClient.new_builder() \
        .with_credentials(credentials) \
        .with_region(MrsRegion.value_of("<YOUR REGION>")) \
        .build()

    try:
        request = CreateClusterRequest()
        listTagsbody = [
            Tag(
                key="key1",
                value="value1"
            ),
            Tag(
                key="key2",
                value="value2"
            )
        ]
        listActionStagesBootstrapScripts = [
            "AFTER_SCALE_IN",
            "AFTER_SCALE_OUT"
        ]
        listNodesBootstrapScripts = [
            "master"
        ]
        listBootstrapScriptsbody = [
            BootstrapScript(
                name="Install zepplin",
                uri="s3a://XXX/zeppelin_install.sh",
                parameters="",
                nodes=listNodesBootstrapScripts,
                active_master=False,
                fail_action="continue",
                before_component_start=False,
                start_time=1667892101,
                state="IN_PROGRESS",
                action_stages=listActionStagesBootstrapScripts
            )
        ]
        listAddJobsbody = [
            AddJobsReqV11(
                job_type=1,
                job_name="tenji111",
                jar_path="s3a://bigdata/program/hadoop-mapreduce-examples-XXX.jar",
                arguments="wordcount",
                input="s3a://bigdata/input/wd_1k/",
                output="s3a://bigdata/ouput/",
                job_log="s3a://bigdata/log/",
                hive_script_path="",
                hql="",
                shutdown_cluster=False,
                submit_job_once_cluster_run=True,
                file_action=""
            )
        ]
        listComponentListbody = [
            ComponentAmbV11(
                component_name="Hadoop"
            ),
            ComponentAmbV11(
                component_name="Spark"
            ),
            ComponentAmbV11(
                component_name="HBase"
            ),
            ComponentAmbV11(
                component_name="Hive"
            ),
            ComponentAmbV11(
                component_name="Presto"
            ),
            ComponentAmbV11(
                component_name="Tez"
            ),
            ComponentAmbV11(
                component_name="Hue"
            ),
            ComponentAmbV11(
                component_name="Loader"
            ),
            ComponentAmbV11(
                component_name="Flink"
            )
        ]
        request.body = CreateClusterReqV11(
            login_mode=1,
            tags=listTagsbody,
            enterprise_project_id="0",
            log_collection=1,
            cluster_type=0,
            safe_mode=0,
            cluster_admin_secret="******",
            node_public_cert_name="SSHkey-bba1",
            bootstrap_scripts=listBootstrapScriptsbody,
            core_data_volume_count=1,
            core_data_volume_size=600,
            core_data_volume_type="SATA",
            master_data_volume_count=1,
            master_data_volume_size=600,
            master_data_volume_type="SATA",
            add_jobs=listAddJobsbody,
            security_groups_id="",
            subnet_name="subnet",
            subnet_id="815bece0-fd22-4b65-8a6e-15788c99ee43",
            vpc_id="5b7db34d-3534-4a6e-ac94-023cd36aaf74",
            available_zone_id="d573142f24894ef3bd3664de068b44b0",
            component_list=listComponentListbody,
            core_node_size="s1.xlarge.linux.bigdata",
            master_node_size="s3.2xlarge.2.linux.bigdata",
            vpc="vpc1",
            data_center="",
            billing_type=12,
            core_node_num=1,
            master_node_num=1,
            cluster_name="newcluster",
            cluster_version="MRS 3.1.0"
        )
        response = client.create_cluster(request)
        print(response)
    except exceptions.ClientRequestException as e:
        print(e.status_code)
        print(e.request_id)
        print(e.error_code)
        print(e.error_msg)

  • Use the node_groups parameter group to create a cluster with the cluster HA feature enabled, with the cluster version of MRS 3.1.0.

      1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
    		 
    package main

import (
	"fmt"
	"github.com/huaweicloud/huaweicloud-sdk-go-v3/core/auth/basic"
    mrs "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/mrs/v1"
	"github.com/huaweicloud/huaweicloud-sdk-go-v3/services/mrs/v1/model"
    region "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/mrs/v1/region"
)

func main() {
    // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
    // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
    ak := os.Getenv("CLOUD_SDK_AK")
    sk := os.Getenv("CLOUD_SDK_SK")
    projectId := "{project_id}"

    auth := basic.NewCredentialsBuilder().
        WithAk(ak).
        WithSk(sk).
        WithProjectId(projectId).
        Build()

    client := mrs.NewMrsClient(
        mrs.MrsClientBuilder().
            WithRegion(region.ValueOf("<YOUR REGION>")).
            WithCredential(auth).
            Build())

    request := &model.CreateClusterRequest{}
	var listNodesExecScripts = []string{
        "master",
	    "core",
	    "task",
    }
	var listNodesExecScripts1 = []string{
        "master",
	    "core",
	    "task",
    }
	parametersExecScripts:= "${mrs_scale_node_num} ${mrs_scale_type} xxx"
	activeMasterExecScripts:= true
	parametersExecScripts1:= "${mrs_scale_node_hostnames} ${mrs_scale_node_ips}"
	activeMasterExecScripts1:= true
	var listExecScriptsAutoScalingPolicy = []model.ScaleScript{
        {
            Name: "before_scale_out",
            Uri: "s3a://XXX/zeppelin_install.sh",
            Parameters: &parametersExecScripts,
            Nodes: listNodesExecScripts1,
            ActiveMaster: &activeMasterExecScripts,
            FailAction: model.GetScaleScriptFailActionEnum().CONTINUE,
            ActionStage: model.GetScaleScriptActionStageEnum().BEFORE_SCALE_OUT,
        },
        {
            Name: "after_scale_out",
            Uri: "s3a://XXX/storm_rebalance.sh",
            Parameters: &parametersExecScripts1,
            Nodes: listNodesExecScripts,
            ActiveMaster: &activeMasterExecScripts1,
            FailAction: model.GetScaleScriptFailActionEnum().CONTINUE,
            ActionStage: model.GetScaleScriptActionStageEnum().AFTER_SCALE_OUT,
        },
    }
	comparisonOperatorTrigger:= "GT"
	triggerRules := &model.Trigger{
		MetricName: "YARNMemoryAvailablePercentage",
		MetricValue: "70",
		ComparisonOperator: &comparisonOperatorTrigger,
		EvaluationPeriods: int32(10),
	}
	comparisonOperatorTrigger1:= "LT"
	triggerRules1 := &model.Trigger{
		MetricName: "YARNMemoryAvailablePercentage",
		MetricValue: "25",
		ComparisonOperator: &comparisonOperatorTrigger1,
		EvaluationPeriods: int32(10),
	}
	var listRulesAutoScalingPolicy = []model.Rule{
        {
            Name: "default-expand-1",
            AdjustmentType: model.GetRuleAdjustmentTypeEnum().SCALE_OUT,
            CoolDownMinutes: int32(5),
            ScalingAdjustment: int32(1),
            Trigger: triggerRules1,
        },
        {
            Name: "default-shrink-1",
            AdjustmentType: model.GetRuleAdjustmentTypeEnum().SCALE_IN,
            CoolDownMinutes: int32(5),
            ScalingAdjustment: int32(1),
            Trigger: triggerRules,
        },
    }
	var listResourcesPlansAutoScalingPolicy = []model.ResourcesPlan{
        {
            PeriodType: "daily",
            StartTime: "9:50",
            EndTime: "10:20",
            MinCapacity: int32(2),
            MaxCapacity: int32(3),
        },
        {
            PeriodType: "daily",
            StartTime: "10:20",
            EndTime: "12:30",
            MinCapacity: int32(0),
            MaxCapacity: int32(2),
        },
    }
	autoScalingPolicyNodeGroups := &model.AutoScalingPolicy{
		AutoScalingEnable: true,
		MinCapacity: int32(1),
		MaxCapacity: int32(3),
		ResourcesPlans: &listResourcesPlansAutoScalingPolicy,
		Rules: &listRulesAutoScalingPolicy,
		ExecScripts: &listExecScriptsAutoScalingPolicy,
	}
	rootVolumeSizeNodeGroups:= "480"
	rootVolumeTypeNodeGroups:= "SATA"
	dataVolumeTypeNodeGroups:= "SATA"
	dataVolumeCountNodeGroups:= int32(1)
	dataVolumeSizeNodeGroups:= int32(600)
	rootVolumeSizeNodeGroups1:= "480"
	rootVolumeTypeNodeGroups1:= "SATA"
	dataVolumeTypeNodeGroups1:= "SATA"
	dataVolumeCountNodeGroups1:= int32(1)
	dataVolumeSizeNodeGroups1:= int32(600)
	rootVolumeSizeNodeGroups2:= "480"
	rootVolumeTypeNodeGroups2:= "SATA"
	dataVolumeTypeNodeGroups2:= "SATA"
	dataVolumeCountNodeGroups2:= int32(0)
	dataVolumeSizeNodeGroups2:= int32(600)
	var listNodeGroupsbody = []model.NodeGroupV11{
        {
            GroupName: "master_node_default_group",
            NodeNum: int32(2),
            NodeSize: "s3.xlarge.2.linux.bigdata",
            RootVolumeSize: &rootVolumeSizeNodeGroups,
            RootVolumeType: &rootVolumeTypeNodeGroups,
            DataVolumeType: &dataVolumeTypeNodeGroups,
            DataVolumeCount: &dataVolumeCountNodeGroups,
            DataVolumeSize: &dataVolumeSizeNodeGroups,
        },
        {
            GroupName: "core_node_analysis_group",
            NodeNum: int32(3),
            NodeSize: "s3.xlarge.2.linux.bigdata",
            RootVolumeSize: &rootVolumeSizeNodeGroups1,
            RootVolumeType: &rootVolumeTypeNodeGroups1,
            DataVolumeType: &dataVolumeTypeNodeGroups1,
            DataVolumeCount: &dataVolumeCountNodeGroups1,
            DataVolumeSize: &dataVolumeSizeNodeGroups1,
        },
        {
            GroupName: "task_node_analysis_group",
            NodeNum: int32(2),
            NodeSize: "s3.xlarge.2.linux.bigdata",
            RootVolumeSize: &rootVolumeSizeNodeGroups2,
            RootVolumeType: &rootVolumeTypeNodeGroups2,
            DataVolumeType: &dataVolumeTypeNodeGroups2,
            DataVolumeCount: &dataVolumeCountNodeGroups2,
            DataVolumeSize: &dataVolumeSizeNodeGroups2,
            AutoScalingPolicy: autoScalingPolicyNodeGroups,
        },
    }
	var listTagsbody = []model.Tag{
        {
            Key: "key1",
            Value: "value1",
        },
        {
            Key: "key2",
            Value: "value2",
        },
    }
	var listActionStagesBootstrapScripts = []model.BootstrapScriptActionStages{
        model.GetBootstrapScriptActionStagesEnum().AFTER_SCALE_IN,
	    model.GetBootstrapScriptActionStagesEnum().AFTER_SCALE_OUT,
    }
	var listNodesBootstrapScripts = []string{
        "master",
    }
	var listActionStagesBootstrapScripts1 = []model.BootstrapScriptActionStages{
        model.GetBootstrapScriptActionStagesEnum().BEFORE_COMPONENT_FIRST_START,
	    model.GetBootstrapScriptActionStagesEnum().BEFORE_SCALE_IN,
    }
	var listNodesBootstrapScripts1 = []string{
        "master",
	    "core",
	    "task",
    }
	parametersBootstrapScripts:= "param1 param2"
	activeMasterBootstrapScripts:= false
	beforeComponentStartBootstrapScripts:= true
	startTimeBootstrapScripts:= int64(1667892101)
	stateBootstrapScripts:= model.GetBootstrapScriptStateEnum().IN_PROGRESS
	parametersBootstrapScripts1:= ""
	activeMasterBootstrapScripts1:= true
	beforeComponentStartBootstrapScripts1:= false
	startTimeBootstrapScripts1:= int64(1667892101)
	stateBootstrapScripts1:= model.GetBootstrapScriptStateEnum().IN_PROGRESS
	var listBootstrapScriptsbody = []model.BootstrapScript{
        {
            Name: "Modify os config",
            Uri: "s3a://XXX/modify_os_config.sh",
            Parameters: &parametersBootstrapScripts,
            Nodes: listNodesBootstrapScripts1,
            ActiveMaster: &activeMasterBootstrapScripts,
            FailAction: model.GetBootstrapScriptFailActionEnum().CONTINUE,
            BeforeComponentStart: &beforeComponentStartBootstrapScripts,
            StartTime: &startTimeBootstrapScripts,
            State: &stateBootstrapScripts,
            ActionStages: &listActionStagesBootstrapScripts1,
        },
        {
            Name: "Install zepplin",
            Uri: "s3a://XXX/zeppelin_install.sh",
            Parameters: &parametersBootstrapScripts1,
            Nodes: listNodesBootstrapScripts,
            ActiveMaster: &activeMasterBootstrapScripts1,
            FailAction: model.GetBootstrapScriptFailActionEnum().CONTINUE,
            BeforeComponentStart: &beforeComponentStartBootstrapScripts1,
            StartTime: &startTimeBootstrapScripts1,
            State: &stateBootstrapScripts1,
            ActionStages: &listActionStagesBootstrapScripts,
        },
    }
	jarPathAddJobs:= "s3a://bigdata/program/hadoop-mapreduce-examples-2.7.2.jar"
	argumentsAddJobs:= "wordcount"
	inputAddJobs:= "s3a://bigdata/input/wd_1k/"
	outputAddJobs:= "s3a://bigdata/ouput/"
	jobLogAddJobs:= "s3a://bigdata/log/"
	hiveScriptPathAddJobs:= ""
	hqlAddJobs:= ""
	shutdownClusterAddJobs:= true
	fileActionAddJobs:= ""
	var listAddJobsbody = []model.AddJobsReqV11{
        {
            JobType: int32(1),
            JobName: "tenji111",
            JarPath: &jarPathAddJobs,
            Arguments: &argumentsAddJobs,
            Input: &inputAddJobs,
            Output: &outputAddJobs,
            JobLog: &jobLogAddJobs,
            HiveScriptPath: &hiveScriptPathAddJobs,
            Hql: &hqlAddJobs,
            ShutdownCluster: &shutdownClusterAddJobs,
            SubmitJobOnceClusterRun: true,
            FileAction: &fileActionAddJobs,
        },
    }
	var listComponentListbody = []model.ComponentAmbV11{
        {
            ComponentName: "Hadoop",
        },
        {
            ComponentName: "Spark",
        },
        {
            ComponentName: "HBase",
        },
        {
            ComponentName: "Hive",
        },
        {
            ComponentName: "Presto",
        },
        {
            ComponentName: "Tez",
        },
        {
            ComponentName: "Hue",
        },
        {
            ComponentName: "Loader",
        },
        {
            ComponentName: "Flink",
        },
    }
	loginModeCreateClusterReqV11:= model.GetCreateClusterReqV11LoginModeEnum().E_1
	enterpriseProjectIdCreateClusterReqV11:= "0"
	logCollectionCreateClusterReqV11:= model.GetCreateClusterReqV11LogCollectionEnum().E_1
	clusterTypeCreateClusterReqV11:= model.GetCreateClusterReqV11ClusterTypeEnum().E_0
	clusterMasterSecretCreateClusterReqV11:= ""
	clusterAdminSecretCreateClusterReqV11:= ""
	securityGroupsIdCreateClusterReqV11:= "4820eace-66ad-4f2c-8d46-cf340e3029dd"
	request.Body = &model.CreateClusterReqV11{
		NodeGroups: &listNodeGroupsbody,
		LoginMode: &loginModeCreateClusterReqV11,
		Tags: &listTagsbody,
		EnterpriseProjectId: &enterpriseProjectIdCreateClusterReqV11,
		LogCollection: &logCollectionCreateClusterReqV11,
		ClusterType: &clusterTypeCreateClusterReqV11,
		SafeMode: model.GetCreateClusterReqV11SafeModeEnum().E_0,
		ClusterMasterSecret: &clusterMasterSecretCreateClusterReqV11,
		ClusterAdminSecret: &clusterAdminSecretCreateClusterReqV11,
		BootstrapScripts: &listBootstrapScriptsbody,
		AddJobs: &listAddJobsbody,
		SecurityGroupsId: &securityGroupsIdCreateClusterReqV11,
		SubnetName: "subnet-4b44",
		SubnetId: "67984709-e15e-4e86-9886-d76712d4e00a",
		VpcId: "4a365717-67be-4f33-80c5-98e98a813af8",
		AvailableZoneId: "d573142f24894ef3bd3664de068b44b0",
		ComponentList: listComponentListbody,
		Vpc: "vpc-4b1c",
		DataCenter: "",
		BillingType: model.GetCreateClusterReqV11BillingTypeEnum().E_12,
		ClusterName: "mrs_HEbK",
		ClusterVersion: "MRS 3.1.0",
	}
	response, err := client.CreateCluster(request)
	if err == nil {
        fmt.Printf("%+v\n", response)
    } else {
        fmt.Println(err)
    }
}

  • Do not use the node_groups parameter group to create a cluster with the cluster HA feature enabled, with the cluster version of MRS 3.1.0.

      1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
    		 
    package main

import (
	"fmt"
	"github.com/huaweicloud/huaweicloud-sdk-go-v3/core/auth/basic"
    mrs "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/mrs/v1"
	"github.com/huaweicloud/huaweicloud-sdk-go-v3/services/mrs/v1/model"
    region "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/mrs/v1/region"
)

func main() {
    // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
    // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
    ak := os.Getenv("CLOUD_SDK_AK")
    sk := os.Getenv("CLOUD_SDK_SK")
    projectId := "{project_id}"

    auth := basic.NewCredentialsBuilder().
        WithAk(ak).
        WithSk(sk).
        WithProjectId(projectId).
        Build()

    client := mrs.NewMrsClient(
        mrs.MrsClientBuilder().
            WithRegion(region.ValueOf("<YOUR REGION>")).
            WithCredential(auth).
            Build())

    request := &model.CreateClusterRequest{}
	var listTagsbody = []model.Tag{
        {
            Key: "key1",
            Value: "value1",
        },
        {
            Key: "key2",
            Value: "value2",
        },
    }
	var listActionStagesBootstrapScripts = []model.BootstrapScriptActionStages{
        model.GetBootstrapScriptActionStagesEnum().AFTER_SCALE_IN,
	    model.GetBootstrapScriptActionStagesEnum().AFTER_SCALE_OUT,
    }
	var listNodesBootstrapScripts = []string{
        "master",
    }
	var listActionStagesBootstrapScripts1 = []model.BootstrapScriptActionStages{
        model.GetBootstrapScriptActionStagesEnum().BEFORE_COMPONENT_FIRST_START,
	    model.GetBootstrapScriptActionStagesEnum().BEFORE_SCALE_IN,
    }
	var listNodesBootstrapScripts1 = []string{
        "master",
	    "core",
	    "task",
    }
	parametersBootstrapScripts:= "param1param2"
	activeMasterBootstrapScripts:= false
	beforeComponentStartBootstrapScripts:= true
	startTimeBootstrapScripts:= int64(1667892101)
	stateBootstrapScripts:= model.GetBootstrapScriptStateEnum().IN_PROGRESS
	parametersBootstrapScripts1:= ""
	activeMasterBootstrapScripts1:= true
	beforeComponentStartBootstrapScripts1:= false
	startTimeBootstrapScripts1:= int64(1667892101)
	stateBootstrapScripts1:= model.GetBootstrapScriptStateEnum().IN_PROGRESS
	var listBootstrapScriptsbody = []model.BootstrapScript{
        {
            Name: "Modifyosconfig",
            Uri: "s3a://XXX/modify_os_config.sh",
            Parameters: &parametersBootstrapScripts,
            Nodes: listNodesBootstrapScripts1,
            ActiveMaster: &activeMasterBootstrapScripts,
            FailAction: model.GetBootstrapScriptFailActionEnum().CONTINUE,
            BeforeComponentStart: &beforeComponentStartBootstrapScripts,
            StartTime: &startTimeBootstrapScripts,
            State: &stateBootstrapScripts,
            ActionStages: &listActionStagesBootstrapScripts1,
        },
        {
            Name: "Installzepplin",
            Uri: "s3a://XXX/zeppelin_install.sh",
            Parameters: &parametersBootstrapScripts1,
            Nodes: listNodesBootstrapScripts,
            ActiveMaster: &activeMasterBootstrapScripts1,
            FailAction: model.GetBootstrapScriptFailActionEnum().CONTINUE,
            BeforeComponentStart: &beforeComponentStartBootstrapScripts1,
            StartTime: &startTimeBootstrapScripts1,
            State: &stateBootstrapScripts1,
            ActionStages: &listActionStagesBootstrapScripts,
        },
    }
	var listNodesExecScripts = []string{
        "master",
	    "core",
	    "task",
    }
	var listNodesExecScripts1 = []string{
        "master",
	    "core",
	    "task",
    }
	parametersExecScripts:= "${mrs_scale_node_num} ${mrs_scale_type}xxx"
	activeMasterExecScripts:= true
	parametersExecScripts1:= "${mrs_scale_node_hostnames} ${mrs_scale_node_ips}"
	activeMasterExecScripts1:= true
	var listExecScriptsAutoScalingPolicy = []model.ScaleScript{
        {
            Name: "before_scale_out",
            Uri: "s3a://XXX/zeppelin_install.sh",
            Parameters: &parametersExecScripts,
            Nodes: listNodesExecScripts1,
            ActiveMaster: &activeMasterExecScripts,
            FailAction: model.GetScaleScriptFailActionEnum().CONTINUE,
            ActionStage: model.GetScaleScriptActionStageEnum().BEFORE_SCALE_OUT,
        },
        {
            Name: "after_scale_out",
            Uri: "s3a://XXX/storm_rebalance.sh",
            Parameters: &parametersExecScripts1,
            Nodes: listNodesExecScripts,
            ActiveMaster: &activeMasterExecScripts1,
            FailAction: model.GetScaleScriptFailActionEnum().CONTINUE,
            ActionStage: model.GetScaleScriptActionStageEnum().AFTER_SCALE_OUT,
        },
    }
	comparisonOperatorTrigger:= "GT"
	triggerRules := &model.Trigger{
		MetricName: "YARNMemoryAvailablePercentage",
		MetricValue: "70",
		ComparisonOperator: &comparisonOperatorTrigger,
		EvaluationPeriods: int32(10),
	}
	comparisonOperatorTrigger1:= "LT"
	triggerRules1 := &model.Trigger{
		MetricName: "YARNMemoryAvailablePercentage",
		MetricValue: "25",
		ComparisonOperator: &comparisonOperatorTrigger1,
		EvaluationPeriods: int32(10),
	}
	var listRulesAutoScalingPolicy = []model.Rule{
        {
            Name: "default-expand-1",
            AdjustmentType: model.GetRuleAdjustmentTypeEnum().SCALE_OUT,
            CoolDownMinutes: int32(5),
            ScalingAdjustment: int32(1),
            Trigger: triggerRules1,
        },
        {
            Name: "default-shrink-1",
            AdjustmentType: model.GetRuleAdjustmentTypeEnum().SCALE_IN,
            CoolDownMinutes: int32(5),
            ScalingAdjustment: int32(1),
            Trigger: triggerRules,
        },
    }
	var listResourcesPlansAutoScalingPolicy = []model.ResourcesPlan{
        {
            PeriodType: "daily",
            StartTime: "9:50",
            EndTime: "10:20",
            MinCapacity: int32(2),
            MaxCapacity: int32(3),
        },
        {
            PeriodType: "daily",
            StartTime: "10:20",
            EndTime: "12:30",
            MinCapacity: int32(0),
            MaxCapacity: int32(2),
        },
    }
	autoScalingPolicyTaskNodeGroups := &model.AutoScalingPolicy{
		AutoScalingEnable: true,
		MinCapacity: int32(1),
		MaxCapacity: int32(3),
		ResourcesPlans: &listResourcesPlansAutoScalingPolicy,
		Rules: &listRulesAutoScalingPolicy,
		ExecScripts: &listExecScriptsAutoScalingPolicy,
	}
	var listTaskNodeGroupsbody = []model.TaskNodeGroup{
        {
            NodeNum: int32(2),
            NodeSize: "s3.xlarge.2.linux.bigdata",
            DataVolumeType: model.GetTaskNodeGroupDataVolumeTypeEnum().SATA,
            DataVolumeCount: int32(1),
            DataVolumeSize: int32(600),
            AutoScalingPolicy: autoScalingPolicyTaskNodeGroups,
        },
    }
	jarPathAddJobs:= "s3a://bigdata/program/hadoop-mapreduce-examples-2.7.2.jar"
	argumentsAddJobs:= "wordcount"
	inputAddJobs:= "s3a://bigdata/input/wd_1k/"
	outputAddJobs:= "s3a://bigdata/ouput/"
	jobLogAddJobs:= "s3a://bigdata/log/"
	hiveScriptPathAddJobs:= ""
	hqlAddJobs:= ""
	shutdownClusterAddJobs:= true
	fileActionAddJobs:= ""
	var listAddJobsbody = []model.AddJobsReqV11{
        {
            JobType: int32(1),
            JobName: "tenji111",
            JarPath: &jarPathAddJobs,
            Arguments: &argumentsAddJobs,
            Input: &inputAddJobs,
            Output: &outputAddJobs,
            JobLog: &jobLogAddJobs,
            HiveScriptPath: &hiveScriptPathAddJobs,
            Hql: &hqlAddJobs,
            ShutdownCluster: &shutdownClusterAddJobs,
            SubmitJobOnceClusterRun: true,
            FileAction: &fileActionAddJobs,
        },
    }
	var listComponentListbody = []model.ComponentAmbV11{
        {
            ComponentName: "Hadoop",
        },
        {
            ComponentName: "Spark",
        },
        {
            ComponentName: "HBase",
        },
        {
            ComponentName: "Hive",
        },
    }
	logCollectionCreateClusterReqV11:= model.GetCreateClusterReqV11LogCollectionEnum().E_1
	clusterTypeCreateClusterReqV11:= model.GetCreateClusterReqV11ClusterTypeEnum().E_0
	nodePublicCertNameCreateClusterReqV11:= "SSHkey-bba1"
	coreDataVolumeCountCreateClusterReqV11:= int32(2)
	coreDataVolumeSizeCreateClusterReqV11:= int32(600)
	coreDataVolumeTypeCreateClusterReqV11:= model.GetCreateClusterReqV11CoreDataVolumeTypeEnum().SATA
	masterDataVolumeCountCreateClusterReqV11:= model.GetCreateClusterReqV11MasterDataVolumeCountEnum().E_1
	masterDataVolumeSizeCreateClusterReqV11:= int32(600)
	masterDataVolumeTypeCreateClusterReqV11:= model.GetCreateClusterReqV11MasterDataVolumeTypeEnum().SATA
	securityGroupsIdCreateClusterReqV11:= "845bece1-fd22-4b45-7a6e-14338c99ee43"
	coreNodeSizeCreateClusterReqV11:= "s1.xlarge.linux.bigdata"
	masterNodeSizeCreateClusterReqV11:= "s3.2xlarge.2.linux.bigdata"
	coreNodeNumCreateClusterReqV11:= int32(3)
	masterNodeNumCreateClusterReqV11:= int32(2)
	request.Body = &model.CreateClusterReqV11{
		Tags: &listTagsbody,
		LogCollection: &logCollectionCreateClusterReqV11,
		ClusterType: &clusterTypeCreateClusterReqV11,
		SafeMode: model.GetCreateClusterReqV11SafeModeEnum().E_0,
		NodePublicCertName: &nodePublicCertNameCreateClusterReqV11,
		BootstrapScripts: &listBootstrapScriptsbody,
		TaskNodeGroups: &listTaskNodeGroupsbody,
		CoreDataVolumeCount: &coreDataVolumeCountCreateClusterReqV11,
		CoreDataVolumeSize: &coreDataVolumeSizeCreateClusterReqV11,
		CoreDataVolumeType: &coreDataVolumeTypeCreateClusterReqV11,
		MasterDataVolumeCount: &masterDataVolumeCountCreateClusterReqV11,
		MasterDataVolumeSize: &masterDataVolumeSizeCreateClusterReqV11,
		MasterDataVolumeType: &masterDataVolumeTypeCreateClusterReqV11,
		AddJobs: &listAddJobsbody,
		SecurityGroupsId: &securityGroupsIdCreateClusterReqV11,
		SubnetName: "subnet",
		SubnetId: "815bece0-fd22-4b65-8a6e-15788c99ee43",
		VpcId: "5b7db34d-3534-4a6e-ac94-023cd36aaf74",
		AvailableZoneId: "d573142f24894ef3bd3664de068b44b0",
		ComponentList: listComponentListbody,
		CoreNodeSize: &coreNodeSizeCreateClusterReqV11,
		MasterNodeSize: &masterNodeSizeCreateClusterReqV11,
		Vpc: "vpc1",
		DataCenter: "",
		BillingType: model.GetCreateClusterReqV11BillingTypeEnum().E_12,
		CoreNodeNum: &coreNodeNumCreateClusterReqV11,
		MasterNodeNum: &masterNodeNumCreateClusterReqV11,
		ClusterName: "newcluster",
		ClusterVersion: "MRS 3.1.0",
	}
	response, err := client.CreateCluster(request)
	if err == nil {
        fmt.Printf("%+v\n", response)
    } else {
        fmt.Println(err)
    }
}

  • Use the node_groups parameter group to create a cluster with the cluster HA feature disabled and the minimum specifications, with the cluster version of MRS 3.1.0.

      1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
    		 
    package main

import (
	"fmt"
	"github.com/huaweicloud/huaweicloud-sdk-go-v3/core/auth/basic"
    mrs "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/mrs/v1"
	"github.com/huaweicloud/huaweicloud-sdk-go-v3/services/mrs/v1/model"
    region "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/mrs/v1/region"
)

func main() {
    // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
    // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
    ak := os.Getenv("CLOUD_SDK_AK")
    sk := os.Getenv("CLOUD_SDK_SK")
    projectId := "{project_id}"

    auth := basic.NewCredentialsBuilder().
        WithAk(ak).
        WithSk(sk).
        WithProjectId(projectId).
        Build()

    client := mrs.NewMrsClient(
        mrs.MrsClientBuilder().
            WithRegion(region.ValueOf("<YOUR REGION>")).
            WithCredential(auth).
            Build())

    request := &model.CreateClusterRequest{}
	rootVolumeSizeNodeGroups:= "480"
	rootVolumeTypeNodeGroups:= "SATA"
	dataVolumeTypeNodeGroups:= "SATA"
	dataVolumeCountNodeGroups:= int32(1)
	dataVolumeSizeNodeGroups:= int32(600)
	rootVolumeSizeNodeGroups1:= "480"
	rootVolumeTypeNodeGroups1:= "SATA"
	dataVolumeTypeNodeGroups1:= "SATA"
	dataVolumeCountNodeGroups1:= int32(1)
	dataVolumeSizeNodeGroups1:= int32(600)
	var listNodeGroupsbody = []model.NodeGroupV11{
        {
            GroupName: "master_node_default_group",
            NodeNum: int32(1),
            NodeSize: "s3.xlarge.2.linux.bigdata",
            RootVolumeSize: &rootVolumeSizeNodeGroups,
            RootVolumeType: &rootVolumeTypeNodeGroups,
            DataVolumeType: &dataVolumeTypeNodeGroups,
            DataVolumeCount: &dataVolumeCountNodeGroups,
            DataVolumeSize: &dataVolumeSizeNodeGroups,
        },
        {
            GroupName: "core_node_analysis_group",
            NodeNum: int32(1),
            NodeSize: "s3.xlarge.2.linux.bigdata",
            RootVolumeSize: &rootVolumeSizeNodeGroups1,
            RootVolumeType: &rootVolumeTypeNodeGroups1,
            DataVolumeType: &dataVolumeTypeNodeGroups1,
            DataVolumeCount: &dataVolumeCountNodeGroups1,
            DataVolumeSize: &dataVolumeSizeNodeGroups1,
        },
    }
	var listTagsbody = []model.Tag{
        {
            Key: "key1",
            Value: "value1",
        },
        {
            Key: "key2",
            Value: "value2",
        },
    }
	var listActionStagesBootstrapScripts = []model.BootstrapScriptActionStages{
        model.GetBootstrapScriptActionStagesEnum().AFTER_SCALE_IN,
	    model.GetBootstrapScriptActionStagesEnum().AFTER_SCALE_OUT,
    }
	var listNodesBootstrapScripts = []string{
        "master",
    }
	var listActionStagesBootstrapScripts1 = []model.BootstrapScriptActionStages{
        model.GetBootstrapScriptActionStagesEnum().BEFORE_COMPONENT_FIRST_START,
	    model.GetBootstrapScriptActionStagesEnum().BEFORE_SCALE_IN,
    }
	var listNodesBootstrapScripts1 = []string{
        "master",
	    "core",
	    "task",
    }
	parametersBootstrapScripts:= "param1 param2"
	activeMasterBootstrapScripts:= false
	beforeComponentStartBootstrapScripts:= true
	startTimeBootstrapScripts:= int64(1667892101)
	stateBootstrapScripts:= model.GetBootstrapScriptStateEnum().IN_PROGRESS
	parametersBootstrapScripts1:= ""
	activeMasterBootstrapScripts1:= true
	beforeComponentStartBootstrapScripts1:= false
	startTimeBootstrapScripts1:= int64(1667892101)
	stateBootstrapScripts1:= model.GetBootstrapScriptStateEnum().IN_PROGRESS
	var listBootstrapScriptsbody = []model.BootstrapScript{
        {
            Name: "Modify os config",
            Uri: "s3a://XXX/modify_os_config.sh",
            Parameters: &parametersBootstrapScripts,
            Nodes: listNodesBootstrapScripts1,
            ActiveMaster: &activeMasterBootstrapScripts,
            FailAction: model.GetBootstrapScriptFailActionEnum().CONTINUE,
            BeforeComponentStart: &beforeComponentStartBootstrapScripts,
            StartTime: &startTimeBootstrapScripts,
            State: &stateBootstrapScripts,
            ActionStages: &listActionStagesBootstrapScripts1,
        },
        {
            Name: "Install zepplin",
            Uri: "s3a://XXX/zeppelin_install.sh",
            Parameters: &parametersBootstrapScripts1,
            Nodes: listNodesBootstrapScripts,
            ActiveMaster: &activeMasterBootstrapScripts1,
            FailAction: model.GetBootstrapScriptFailActionEnum().CONTINUE,
            BeforeComponentStart: &beforeComponentStartBootstrapScripts1,
            StartTime: &startTimeBootstrapScripts1,
            State: &stateBootstrapScripts1,
            ActionStages: &listActionStagesBootstrapScripts,
        },
    }
	jarPathAddJobs:= "s3a://bigdata/program/hadoop-mapreduce-examples-2.7.2.jar"
	argumentsAddJobs:= "wordcount"
	inputAddJobs:= "s3a://bigdata/input/wd_1k/"
	outputAddJobs:= "s3a://bigdata/ouput/"
	jobLogAddJobs:= "s3a://bigdata/log/"
	hiveScriptPathAddJobs:= ""
	hqlAddJobs:= ""
	shutdownClusterAddJobs:= true
	fileActionAddJobs:= ""
	var listAddJobsbody = []model.AddJobsReqV11{
        {
            JobType: int32(1),
            JobName: "tenji111",
            JarPath: &jarPathAddJobs,
            Arguments: &argumentsAddJobs,
            Input: &inputAddJobs,
            Output: &outputAddJobs,
            JobLog: &jobLogAddJobs,
            HiveScriptPath: &hiveScriptPathAddJobs,
            Hql: &hqlAddJobs,
            ShutdownCluster: &shutdownClusterAddJobs,
            SubmitJobOnceClusterRun: true,
            FileAction: &fileActionAddJobs,
        },
    }
	var listComponentListbody = []model.ComponentAmbV11{
        {
            ComponentName: "Hadoop",
        },
        {
            ComponentName: "Spark",
        },
        {
            ComponentName: "HBase",
        },
        {
            ComponentName: "Hive",
        },
        {
            ComponentName: "Presto",
        },
        {
            ComponentName: "Tez",
        },
        {
            ComponentName: "Hue",
        },
        {
            ComponentName: "Loader",
        },
        {
            ComponentName: "Flink",
        },
    }
	loginModeCreateClusterReqV11:= model.GetCreateClusterReqV11LoginModeEnum().E_1
	enterpriseProjectIdCreateClusterReqV11:= "0"
	logCollectionCreateClusterReqV11:= model.GetCreateClusterReqV11LogCollectionEnum().E_1
	clusterTypeCreateClusterReqV11:= model.GetCreateClusterReqV11ClusterTypeEnum().E_0
	clusterMasterSecretCreateClusterReqV11:= ""
	clusterAdminSecretCreateClusterReqV11:= ""
	securityGroupsIdCreateClusterReqV11:= "4820eace-66ad-4f2c-8d46-cf340e3029dd"
	request.Body = &model.CreateClusterReqV11{
		NodeGroups: &listNodeGroupsbody,
		LoginMode: &loginModeCreateClusterReqV11,
		Tags: &listTagsbody,
		EnterpriseProjectId: &enterpriseProjectIdCreateClusterReqV11,
		LogCollection: &logCollectionCreateClusterReqV11,
		ClusterType: &clusterTypeCreateClusterReqV11,
		SafeMode: model.GetCreateClusterReqV11SafeModeEnum().E_0,
		ClusterMasterSecret: &clusterMasterSecretCreateClusterReqV11,
		ClusterAdminSecret: &clusterAdminSecretCreateClusterReqV11,
		BootstrapScripts: &listBootstrapScriptsbody,
		AddJobs: &listAddJobsbody,
		SecurityGroupsId: &securityGroupsIdCreateClusterReqV11,
		SubnetName: "subnet-4b44",
		SubnetId: "67984709-e15e-4e86-9886-d76712d4e00a",
		VpcId: "4a365717-67be-4f33-80c5-98e98a813af8",
		AvailableZoneId: "d573142f24894ef3bd3664de068b44b0",
		ComponentList: listComponentListbody,
		Vpc: "vpc-4b1c",
		DataCenter: "",
		BillingType: model.GetCreateClusterReqV11BillingTypeEnum().E_12,
		ClusterName: "mrs_HEbK",
		ClusterVersion: "MRS 3.1.0",
	}
	response, err := client.CreateCluster(request)
	if err == nil {
        fmt.Printf("%+v\n", response)
    } else {
        fmt.Println(err)
    }
}

  • Do not use the node_groups parameter group to create a cluster with the cluster HA feature disabled and the minimum specifications, with the cluster version of MRS 3.1.0.

      1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
    		 
    package main

import (
	"fmt"
	"github.com/huaweicloud/huaweicloud-sdk-go-v3/core/auth/basic"
    mrs "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/mrs/v1"
	"github.com/huaweicloud/huaweicloud-sdk-go-v3/services/mrs/v1/model"
    region "github.com/huaweicloud/huaweicloud-sdk-go-v3/services/mrs/v1/region"
)

func main() {
    // The AK and SK used for authentication are hard-coded or stored in plaintext, which has great security risks. It is recommended that the AK and SK be stored in ciphertext in configuration files or environment variables and decrypted during use to ensure security.
    // In this example, AK and SK are stored in environment variables for authentication. Before running this example, set environment variables CLOUD_SDK_AK and CLOUD_SDK_SK in the local environment
    ak := os.Getenv("CLOUD_SDK_AK")
    sk := os.Getenv("CLOUD_SDK_SK")
    projectId := "{project_id}"

    auth := basic.NewCredentialsBuilder().
        WithAk(ak).
        WithSk(sk).
        WithProjectId(projectId).
        Build()

    client := mrs.NewMrsClient(
        mrs.MrsClientBuilder().
            WithRegion(region.ValueOf("<YOUR REGION>")).
            WithCredential(auth).
            Build())

    request := &model.CreateClusterRequest{}
	var listTagsbody = []model.Tag{
        {
            Key: "key1",
            Value: "value1",
        },
        {
            Key: "key2",
            Value: "value2",
        },
    }
	var listActionStagesBootstrapScripts = []model.BootstrapScriptActionStages{
        model.GetBootstrapScriptActionStagesEnum().AFTER_SCALE_IN,
	    model.GetBootstrapScriptActionStagesEnum().AFTER_SCALE_OUT,
    }
	var listNodesBootstrapScripts = []string{
        "master",
    }
	parametersBootstrapScripts:= ""
	activeMasterBootstrapScripts:= false
	beforeComponentStartBootstrapScripts:= false
	startTimeBootstrapScripts:= int64(1667892101)
	stateBootstrapScripts:= model.GetBootstrapScriptStateEnum().IN_PROGRESS
	var listBootstrapScriptsbody = []model.BootstrapScript{
        {
            Name: "Install zepplin",
            Uri: "s3a://XXX/zeppelin_install.sh",
            Parameters: &parametersBootstrapScripts,
            Nodes: listNodesBootstrapScripts,
            ActiveMaster: &activeMasterBootstrapScripts,
            FailAction: model.GetBootstrapScriptFailActionEnum().CONTINUE,
            BeforeComponentStart: &beforeComponentStartBootstrapScripts,
            StartTime: &startTimeBootstrapScripts,
            State: &stateBootstrapScripts,
            ActionStages: &listActionStagesBootstrapScripts,
        },
    }
	jarPathAddJobs:= "s3a://bigdata/program/hadoop-mapreduce-examples-XXX.jar"
	argumentsAddJobs:= "wordcount"
	inputAddJobs:= "s3a://bigdata/input/wd_1k/"
	outputAddJobs:= "s3a://bigdata/ouput/"
	jobLogAddJobs:= "s3a://bigdata/log/"
	hiveScriptPathAddJobs:= ""
	hqlAddJobs:= ""
	shutdownClusterAddJobs:= false
	fileActionAddJobs:= ""
	var listAddJobsbody = []model.AddJobsReqV11{
        {
            JobType: int32(1),
            JobName: "tenji111",
            JarPath: &jarPathAddJobs,
            Arguments: &argumentsAddJobs,
            Input: &inputAddJobs,
            Output: &outputAddJobs,
            JobLog: &jobLogAddJobs,
            HiveScriptPath: &hiveScriptPathAddJobs,
            Hql: &hqlAddJobs,
            ShutdownCluster: &shutdownClusterAddJobs,
            SubmitJobOnceClusterRun: true,
            FileAction: &fileActionAddJobs,
        },
    }
	var listComponentListbody = []model.ComponentAmbV11{
        {
            ComponentName: "Hadoop",
        },
        {
            ComponentName: "Spark",
        },
        {
            ComponentName: "HBase",
        },
        {
            ComponentName: "Hive",
        },
        {
            ComponentName: "Presto",
        },
        {
            ComponentName: "Tez",
        },
        {
            ComponentName: "Hue",
        },
        {
            ComponentName: "Loader",
        },
        {
            ComponentName: "Flink",
        },
    }
	loginModeCreateClusterReqV11:= model.GetCreateClusterReqV11LoginModeEnum().E_1
	enterpriseProjectIdCreateClusterReqV11:= "0"
	logCollectionCreateClusterReqV11:= model.GetCreateClusterReqV11LogCollectionEnum().E_1
	clusterTypeCreateClusterReqV11:= model.GetCreateClusterReqV11ClusterTypeEnum().E_0
	clusterAdminSecretCreateClusterReqV11:= "******"
	nodePublicCertNameCreateClusterReqV11:= "SSHkey-bba1"
	coreDataVolumeCountCreateClusterReqV11:= int32(1)
	coreDataVolumeSizeCreateClusterReqV11:= int32(600)
	coreDataVolumeTypeCreateClusterReqV11:= model.GetCreateClusterReqV11CoreDataVolumeTypeEnum().SATA
	masterDataVolumeCountCreateClusterReqV11:= model.GetCreateClusterReqV11MasterDataVolumeCountEnum().E_1
	masterDataVolumeSizeCreateClusterReqV11:= int32(600)
	masterDataVolumeTypeCreateClusterReqV11:= model.GetCreateClusterReqV11MasterDataVolumeTypeEnum().SATA
	securityGroupsIdCreateClusterReqV11:= ""
	coreNodeSizeCreateClusterReqV11:= "s1.xlarge.linux.bigdata"
	masterNodeSizeCreateClusterReqV11:= "s3.2xlarge.2.linux.bigdata"
	coreNodeNumCreateClusterReqV11:= int32(1)
	masterNodeNumCreateClusterReqV11:= int32(1)
	request.Body = &model.CreateClusterReqV11{
		LoginMode: &loginModeCreateClusterReqV11,
		Tags: &listTagsbody,
		EnterpriseProjectId: &enterpriseProjectIdCreateClusterReqV11,
		LogCollection: &logCollectionCreateClusterReqV11,
		ClusterType: &clusterTypeCreateClusterReqV11,
		SafeMode: model.GetCreateClusterReqV11SafeModeEnum().E_0,
		ClusterAdminSecret: &clusterAdminSecretCreateClusterReqV11,
		NodePublicCertName: &nodePublicCertNameCreateClusterReqV11,
		BootstrapScripts: &listBootstrapScriptsbody,
		CoreDataVolumeCount: &coreDataVolumeCountCreateClusterReqV11,
		CoreDataVolumeSize: &coreDataVolumeSizeCreateClusterReqV11,
		CoreDataVolumeType: &coreDataVolumeTypeCreateClusterReqV11,
		MasterDataVolumeCount: &masterDataVolumeCountCreateClusterReqV11,
		MasterDataVolumeSize: &masterDataVolumeSizeCreateClusterReqV11,
		MasterDataVolumeType: &masterDataVolumeTypeCreateClusterReqV11,
		AddJobs: &listAddJobsbody,
		SecurityGroupsId: &securityGroupsIdCreateClusterReqV11,
		SubnetName: "subnet",
		SubnetId: "815bece0-fd22-4b65-8a6e-15788c99ee43",
		VpcId: "5b7db34d-3534-4a6e-ac94-023cd36aaf74",
		AvailableZoneId: "d573142f24894ef3bd3664de068b44b0",
		ComponentList: listComponentListbody,
		CoreNodeSize: &coreNodeSizeCreateClusterReqV11,
		MasterNodeSize: &masterNodeSizeCreateClusterReqV11,
		Vpc: "vpc1",
		DataCenter: "",
		BillingType: model.GetCreateClusterReqV11BillingTypeEnum().E_12,
		CoreNodeNum: &coreNodeNumCreateClusterReqV11,
		MasterNodeNum: &masterNodeNumCreateClusterReqV11,
		ClusterName: "newcluster",
		ClusterVersion: "MRS 3.1.0",
	}
	response, err := client.CreateCluster(request)
	if err == nil {
        fmt.Printf("%+v\n", response)
    } else {
        fmt.Println(err)
    }
}

For SDK sample code of more programming languages, see the Sample Code tab in API Explorer. SDK sample code can be automatically generated.

Status Codes

Table 16 Status code

Status Code

Description

200

The cluster has been created.

Error Codes

See Error Codes.

Parent Topic: Cluster Management APIs