Help Center/ MapReduce Service/ API Reference/ API V1.1/ Version Metadata/ Querying the Metadata of a Cluster Version

Updated on 2023-12-19 GMT+08:00

View PDF

Querying the Metadata of a Cluster Version

Function

This API is used to query the metadata of a cluster version. If the cluster ID is specified, the latest metadata of the cluster that has been patched can be queried.

Debugging

You can debug this API in API Explorer. Automatic authentication is supported. API Explorer can automatically generate sample SDK code and support sample SDK code debugging.

URI

GET /v1.1/{project_id}/metadata/versions/{version_name}

**Table 1** URI parameters
Parameter	Mandatory	Type	Description
project_id	Yes	String	Project ID
version_name	Yes	String	The cluster version, for example: MRS 3.1.0. If the request client does not support automatic escape, escape the space to %20, for example, MRS %203.1.0.

**Table 2** Query parameters
Parameter	Mandatory	Type	Description
cluster_id	No	String	Cluster ID. If this parameter is specified, the latest metadata of the cluster that has been patched will be obtained.

Request Parameters

None

Response Parameters

Status code: 200

**Table 3** Response body parameter
Parameter	Type	Description
other	Map<String,Object>	Others
name	String	Image version
template_type	String	Template type
image_id	String	Image ID
status	String	Version status
features	Array of strings	Features
cluster_types	Array of strings	Cluster types
version_type	String	Version type
components	Array of VersionComponent objects	Components. For details, see Table 4.
resource_requirement	Array of strings	Description of required resources, such as IP addresses
constraints	VersionConstraint object	Version restrictions. For details about the parameters, see Table 6.
flavors	FlavorLists object	Specifications. For details, see Table 9.
role_deploy_meta	Array of RoleDeployMeta objects	Role deployment policy of component instances. For details, see Table 10.

**Table 4** VersionComponent
Parameter	Type	Description
other	Map<String,Object>	Others
name	String	Component name
version	String	Supported versions
depend_on	Array of strings	Component dependencies
description	String	Component description
available_cluster_types	Array of strings	Cluster types that support the component
external_datasources	Array of ComponentExternalDatasource objects	External data source. For details, see Table 5.
resource_requirement	Array of strings	Description of required resources, such as IP addresses
valid_roles	Array of strings	Valid roles
visible	Boolean	Whether the component is visible
children_components	Array of strings	Sub-components
multi_az_support_status	String	Whether multi-AZ is supported

**Table 5** ComponentExternalDatasource
Parameter	Type	Description
name	String	Name of the external data source
types	Array of strings	Type of the external data source

**Table 6** VersionConstraint
Parameter	Type	Description
other	Map<String,Object>	Other restrictions
node_constraint	NodeConstraints object	Node restrictions. For details, see Table 7.
safe_mode_kerberos_exclude_components	Array of strings	Excluded components in security mode (Kerberos authentication enabled)

**Table 7** NodeConstraints
Parameter	Type	Description
other	Map<String,Object>	Restrictions on other nodes, including the number of nodes and disks
master	NodeConstraint object	Master node restrictions. For details, see Table 8.
core	NodeConstraint object	Core node restrictions. For details, see Table 8.
task	NodeConstraint object	Task node restrictions. For details, see Table 8.
core_separate	NodeConstraint object	Core node restrictions when management and control nodes are different nodes. For details, see Table 8.
core_combine	NodeConstraint object	Core node restrictions when management and control nodes are different nodes. For details, see Table 8.
task_separate	NodeConstraint object	Task node restrictions when management and control nodes are different nodes. For details, see Table 8.
task_combine	NodeConstraint object	Task node restrictions when management and control nodes are different nodes. For details, see Table 8.
node_group_task	NodeConstraint object	Task node group restrictions. For details, see Table 8.

**Table 8** NodeConstraint
Parameter	Type	Description
other	Map<String,Object>	Other restrictions
min_node_num	Integer	Minimum number of nodes
max_node_num	Integer	Minimum number of nodes
min_core_num	Map<String,Integer>	Minimum number of cores
min_mem_size	Map<String,Integer>	Minimum memory
min_disk_size	Integer	Minimum disk capacity
max_node_group_num	Integer	Maximum node goups
min_data_volume_total_size	Map<String,Integer>	Minimum data volume capacity
disk_type_constraint	Map<String,String>	Disk type limitation
min_root_disk_size	Integer	Minimum root disk size

**Table 9** FlavorLists
Parameter	Type	Description
master	Array of strings	Specifications supported by Master nodes
core	Array of strings	Specifications supported by Core nodes
task	Array of strings	Specifications supported by Task nodes

**Table 10** RoleDeployMeta
Parameter	Type	Description
other	Map<String,Object>	Others
name	String	Role name
code_name	String	Role name
component	String	Component to which the role belongs
node_preference	String	Node deployment preference
count	String	Maximum number of roles
affinity	String	Affinity
affinity_target	String	Objectives of affinity
multi_instance	Integer	Multi-instance
role_kind	String	Role type
constraints	Array of strings	Role scope
multi_az_placement	String	Multi-AZ deployment
arbitration_deployment	Boolean	Arbitration deployment
support_elb	Boolean	Whether ELB is supported
multi_affinity_group_enable	Boolean	Whether the multi-affinity group is enabled
local_disks_anti_affinity	Boolean	Local disk anti-affinity
multi_instance_name_pattern	String	Multi-instance name pattern
private_ip	String	Private IP address
weight	String	Weight

Example Request

Obtain the metadata of MRS 3.1.0

GET /v1.1/{project_id}/metadata/versions/MRS%203.1.0

Example Response

Status code: 200

Version metadata details. Some contents are omitted in this example. For details, call the API.

{
  "other" : { },
  "name" : "MRS 3.1.0",
  "template_type" : "clusterFusion",
  "image_id" : "ECS:X86:deda7c89-6069-46d3-af7c-4afc31eec0d9,ECS:ARM:d5f0087b-9fd7-4977-b9c8-d07262f452d6,BMS:ks1ne:bf071a00-d1d6-472f-943b-22f733248f36,BMS:d2:a9a1a279-85c9-4c05-bd0a-4cb786937a09",
  "status" : "GA",
  "features" : [ "anti_affinity", "auto_scaling", "bootstrap_scripts", "bootstrap_support_obs", "custom_config", "custom_topology", "datasource_manager", "decouple_install", "elastic_ip", "fi_cluster", "fi_cluster_v2", "iam_user_sync", "log_collect", "log_collect_v2", "master_scale_up", "mrs_ecs_agency", "multi_disks", "multi_login_mode", "obs_integration", "obs_user_policy", "opensource_port_matrix", "ops_channel", "password_policy_v2", "random_passwd_for_install", "register_dns_server", "resize", "resources_plans", "safe_mode", "scale_scripts", "security_job_submit", "smn_alarm", "spark_sql_validation_in_executor", "support_obs_protocol", "task_node", "unified_management_ui", "sql_execution", "multi_az", "weak_multi_az_deployment_constraint", "multi_master_scale_up", "omm_random_pwd", "metric_monitor", "auth_manager", "cluster_patch", "hw_domain_name", "fi_custom_config", "force_scale_up", "skip_fi_evs_expand", "detach_node" ],
  "cluster_types" : [ "analysis", "streaming", "mixed", "custom" ],
  "version_type" : "basic",
  "components" : [ {
    "other" : { },
    "name" : "Hadoop",
    "version" : "3.1.1",
    "depend_on" : [ "ZooKeeper", "Ranger" ],
    "description" : "A framework that allows for the distributed processing of large data sets across clusters.",
    "available_cluster_types" : [ "analysis", "mixed", "custom" ],
    "external_datasources" : null,
    "resource_requirement" : [ "privateIp:1" ],
    "valid_roles" : [ "NameNode", "Zkfc", "JournalNode", "DataNode", "ResourceManager", "NodeManager", "JobHistoryServer", "TimelineServer", "HttpFS" ],
    "visible" : true,
    "children_components" : [ "HDFS", "Mapreduce", "Yarn" ],
    "multi_az_support_status" : "multi_az_ha"
  }, {
    "other" : { },
    "name" : "Spark2x",
    "version" : "2.4.5",
    "depend_on" : [ "HDFS", "Yarn", "Hive", "KrbClient", "KrbServer", "ZooKeeper" ],
    "description" : "Apache Spark2x is a fast and general engine based on open source Spark2.x for large-scale data processing.",
    "available_cluster_types" : [ "analysis", "mixed", "custom" ],
    "external_datasources" : null,
    "resource_requirement" : null,
    "valid_roles" : [ "JobHistory2x", "JDBCServer2x", "SparkResource2x", "IndexServer2x" ],
    "visible" : true,
    "children_components" : [ "Spark2x" ],
    "multi_az_support_status" : "multi_az_ha"
  }, {
    "other" : { },
    "name" : "HBase",
    "version" : "2.2.3",
    "depend_on" : [ "HDFS", "Yarn", "KrbServer", "ZooKeeper" ],
    "description" : "A scalable, distributed database that supports structured data storage for large tables.",
    "available_cluster_types" : [ "analysis", "mixed", "custom" ],
    "external_datasources" : null,
    "resource_requirement" : null,
    "valid_roles" : [ "HMaster", "RegionServer", "ThriftServer", "Thrift1Server", "RESTServer", "RegionServer_1" ],
    "visible" : true,
    "children_components" : [ "HBase" ],
    "multi_az_support_status" : "multi_az_ha"
  }, {
    "other" : { },
    "name" : "Hive",
    "version" : "3.1.0",
    "depend_on" : [ "HDFS", "Mapreduce", "Yarn", "DBService", "ZooKeeper" ],
    "description" : "A data warehouse infrastructure that provides data summarization and ad hoc querying.",
    "available_cluster_types" : [ "analysis", "mixed", "custom" ],
    "external_datasources" : [ {
      "name" : "hive_metastore",
      "types" : [ "RDS_MYSQL", "DLCATALOG" ]
    } ],
    "resource_requirement" : null,
    "valid_roles" : [ "MetaStore", "WebHCat", "HiveServer" ],
    "visible" : true,
    "children_components" : [ "Hive" ],
    "multi_az_support_status" : "multi_az_ha"
  }, {
    "other" : { },
    "name" : "ZooKeeper",
    "version" : null,
    "depend_on" : [ "KrbClient", "KrbServer" ],
    "description" : null,
    "available_cluster_types" : [ "analysis", "streaming", "mixed", "custom" ],
    "external_datasources" : null,
    "resource_requirement" : null,
    "valid_roles" : [ "quorumpeer" ],
    "visible" : true,
    "children_components" : [ "ZooKeeper" ],
    "multi_az_support_status" : "multi_az_ha"
  } ],
  "resource_requirement" : [ "privateIp:2" ],
  "constraints" : {
    "other" : { },
    "node_constraint" : {
      "other" : { },
      "master" : {
        "other" : { },
        "min_node_num" : 2,
        "max_node_num" : 9,
        "min_core_num" : null,
        "min_mem_size" : null,
        "min_disk_size" : 600,
        "max_node_group_num" : 1,
        "min_data_volume_total_size" : null,
        "disk_type_constraint" : null,
        "min_root_disk_size" : 480
      },
      "core" : {
        "other" : { },
        "min_node_num" : 3,
        "max_node_num" : 500,
        "min_core_num" : null,
        "min_mem_size" : null,
        "min_disk_size" : 600,
        "max_node_group_num" : null,
        "min_data_volume_total_size" : null,
        "disk_type_constraint" : null,
        "min_root_disk_size" : 480
      },
      "task" : null,
      "core_separate" : null,
      "core_combine" : null,
      "task_separate" : null,
      "task_combine" : null,
      "node_group_task" : {
        "other" : { },
        "min_node_num" : 2,
        "max_node_num" : 10000,
        "min_core_num" : null,
        "min_mem_size" : null,
        "min_disk_size" : 600,
        "max_node_group_num" : 9,
        "min_data_volume_total_size" : null,
        "disk_type_constraint" : null,
        "min_root_disk_size" : 480
      }
    },
    "safe_mode_kerberos_exclude_components" : [ "Presto", "Pulsar" ]
  },
  "flavors" : {
    "master" : [ "ac7.4xlarge.4", "ac7.8xlarge.4", "ac7.16xlarge.4", "ac7.32xlarge.4", "ac7.8xlarge.2", "ac7.16xlarge.2", "ac7.32xlarge.2", "am7.2xlarge.8", "am7.4xlarge.8", "am7.8xlarge.8", "c3ne.4xlarge.4", "c3ne.8xlarge.4", "c3ne.15xlarge.4" ],
    "core" : [ "ac7.4xlarge.4", "ac7.8xlarge.4", "ac7.16xlarge.4", "ac7.32xlarge.4", "ac7.8xlarge.2", "ac7.16xlarge.2", "ac7.32xlarge.2", "am7.2xlarge.8", "am7.4xlarge.8", "am7.8xlarge.8", "c3ne.4xlarge.4", "c3ne.8xlarge.4", "c3ne.15xlarge.4", "c6.4xlarge.4", "c6.8xlarge.2", "c6.8xlarge.4", "c6.16xlarge.2" ],
    "task" : [ "ac7.4xlarge.4", "ac7.8xlarge.4", "ac7.16xlarge.4", "ac7.32xlarge.4", "ac7.8xlarge.2", "ac7.16xlarge.2", "ac7.32xlarge.2", "am7.2xlarge.8", "am7.4xlarge.8", "am7.8xlarge.8", "c3ne.4xlarge.4", "c3ne.8xlarge.4", "c3ne.15xlarge.4" ]
  },
  "role_deploy_meta" : [ {
    "other" : { },
    "name" : "OMSServer",
    "code_name" : "OMS",
    "component" : "OMSServer",
    "node_preference" : "MASTER",
    "count" : "2",
    "affinity" : null,
    "affinity_target" : null,
    "multi_instance" : null,
    "role_kind" : null,
    "constraints" : null,
    "multi_az_placement" : "AT_LEAST_2",
    "arbitration_deployment" : false,
    "support_elb" : false,
    "multi_affinity_group_enable" : false,
    "local_disks_anti_affinity" : false,
    "multi_instance_name_pattern" : null,
    "private_ip" : null,
    "weight" : null
  }, {
    "other" : { },
    "name" : "NameNode",
    "code_name" : "NN",
    "component" : "HDFS",
    "node_preference" : "MASTER",
    "count" : "2",
    "affinity" : null,
    "affinity_target" : null,
    "multi_instance" : null,
    "role_kind" : null,
    "constraints" : null,
    "multi_az_placement" : "AT_LEAST_2",
    "arbitration_deployment" : false,
    "support_elb" : false,
    "multi_affinity_group_enable" : false,
    "local_disks_anti_affinity" : false,
    "multi_instance_name_pattern" : null,
    "private_ip" : null,
    "weight" : null
  }, {
    "other" : { },
    "name" : "Zkfc",
    "code_name" : "ZKFC",
    "component" : "HDFS",
    "node_preference" : null,
    "count" : null,
    "affinity" : "JUST_COLOCATE",
    "affinity_target" : "NameNode",
    "multi_instance" : null,
    "role_kind" : null,
    "constraints" : null,
    "multi_az_placement" : "AT_LEAST_2",
    "arbitration_deployment" : false,
    "support_elb" : false,
    "multi_affinity_group_enable" : false,
    "local_disks_anti_affinity" : false,
    "multi_instance_name_pattern" : null,
    "private_ip" : null,
    "weight" : null
  }, {
    "other" : { },
    "name" : "HttpFS",
    "code_name" : "HFS",
    "component" : "HDFS",
    "node_preference" : "MASTER",
    "count" : "[0-10]",
    "affinity" : null,
    "affinity_target" : null,
    "multi_instance" : null,
    "role_kind" : null,
    "constraints" : null,
    "multi_az_placement" : "AT_LEAST_2",
    "arbitration_deployment" : false,
    "support_elb" : false,
    "multi_affinity_group_enable" : false,
    "local_disks_anti_affinity" : false,
    "multi_instance_name_pattern" : null,
    "private_ip" : null,
    "weight" : null
  }, {
    "other" : { },
    "name" : "JournalNode",
    "code_name" : "JN",
    "component" : "HDFS",
    "node_preference" : "MASTER",
    "count" : "[3-60],step=2",
    "affinity" : null,
    "affinity_target" : null,
    "multi_instance" : null,
    "role_kind" : null,
    "constraints" : null,
    "multi_az_placement" : "QUORUM_LIKE",
    "arbitration_deployment" : true,
    "support_elb" : false,
    "multi_affinity_group_enable" : false,
    "local_disks_anti_affinity" : false,
    "multi_instance_name_pattern" : null,
    "private_ip" : null,
    "weight" : null
  }, {
    "other" : { },
    "name" : "DataNode",
    "code_name" : "DN",
    "component" : "HDFS",
    "node_preference" : "NO_LIMIT",
    "count" : "[3-10000]",
    "affinity" : null,
    "affinity_target" : null,
    "multi_instance" : null,
    "role_kind" : "stateful",
    "constraints" : null,
    "multi_az_placement" : "AT_LEAST_2",
    "arbitration_deployment" : false,
    "support_elb" : false,
    "multi_affinity_group_enable" : false,
    "local_disks_anti_affinity" : false,
    "multi_instance_name_pattern" : null,
    "private_ip" : null,
    "weight" : null
  }, {
    "other" : { },
    "name" : "ResourceManager",
    "code_name" : "RM",
    "component" : "Yarn",
    "node_preference" : "MASTER",
    "count" : "2",
    "affinity" : null,
    "affinity_target" : null,
    "multi_instance" : null,
    "role_kind" : null,
    "constraints" : null,
    "multi_az_placement" : "AT_LEAST_2",
    "arbitration_deployment" : false,
    "support_elb" : false,
    "multi_affinity_group_enable" : false,
    "local_disks_anti_affinity" : false,
    "multi_instance_name_pattern" : null,
    "private_ip" : null,
    "weight" : null
  }, {
    "other" : { },
    "name" : "NodeManager",
    "code_name" : "NM",
    "component" : "Yarn",
    "node_preference" : "NO_LIMIT",
    "count" : "[3-10000]",
    "affinity" : null,
    "affinity_target" : null,
    "multi_instance" : null,
    "role_kind" : "stateless",
    "constraints" : null,
    "multi_az_placement" : "AT_LEAST_2",
    "arbitration_deployment" : false,
    "support_elb" : false,
    "multi_affinity_group_enable" : false,
    "local_disks_anti_affinity" : false,
    "multi_instance_name_pattern" : null,
    "private_ip" : null,
    "weight" : null
  } ]
}