更新时间：2024-06-06 GMT+08:00
YARN REST API接口介绍
功能简介
通过HTTP REST API来查看更多Yarn任务的信息。目前Yarn的REST接口只能进行一些资源或者任务的查询。完整和详细的接口请直接参考官网上的描述以了解其使用：http://hadoop.apache.org/docs/r3.1.1/hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html
准备运行环境
- 在节点上安装客户端，例如安装到“/opt/client”目录。
- 进入客户端安装目录“/opt/client”，执行下列命令初始化环境变量。
source bigdata_env
kinit 组件业务用户
kinit一次票据时效24小时。24小时后再次运行样例，需要重新kinit。
- 与HTTP服务访问相比，以HTTPS方式访问Yarn时，由于使用了SSL安全加密，需要确保Curl命令所支持的SSL协议在集群中已添加支持。若不支持，可对应修改集群中SSL协议。例如，若Curl仅支持TLSv1协议，修改方法如下：
登录FusionInsight Manager页面，选择“集群 > 待操作集群的名称 > 服务 > Yarn > 配置 > 全部配置”，在“搜索”框里搜索“hadoop.ssl.enabled.protocols”，查看参数值是否包含“TLSv1”，若不包含，则在配置项“hadoop.ssl.enabled.protocols”中追加“,TLSv1”。清空“ssl.server.exclude.cipher.list”配置项的值，否则以HTTPS访问不了Yarn。单击“保存”，单击“确定”，保存完成后重启该服务。
TLSv1协议存在安全漏洞，请谨慎使用。
操作步骤
- 获取运行在Yarn上的任务的具体信息。
- 命令：
curl -k -i --negotiate -u : "https://10-120-85-2:8090/ws/v1/cluster/apps/"
其中10-120-85-2为ResourceManager主节点的hostname，8090为ResourceManager的端口号。
- 用户能看到哪个队列的任务，要看这个用户是否有这个队列的admin权限。
如果当前组件使用了Ranger进行权限控制，需基于Ranger配置相关策略进行权限管理。
- 运行结果：
{ "apps": { "app": [ { "id": "application_1461743120947_0001", "user": "spark", "name": "Spark-JDBCServer", "queue": "default", "state": "RUNNING", "finalStatus": "UNDEFINED", "progress": 10, "trackingUI": "ApplicationMaster", "trackingUrl": "https://10-120-85-2:8090/proxy/application_1461743120947_0001/", "diagnostics": "AM is launched. ", "clusterId": 1461743120947, "applicationType": "SPARK", "applicationTags": "", "startedTime": 1461804906260, "finishedTime": 0, "elapsedTime": 6888848, "amContainerLogs": "https://10-120-85-2:8044/node/containerlogs/container_e12_1461743120947_0001_01_000001/spark", "amHostHttpAddress": "10-120-85-2:8044", "allocatedMB": 1024, "allocatedVCores": 1, "runningContainers": 1, "memorySeconds": 7053309, "vcoreSeconds": 6887, "preemptedResourceMB": 0, "preemptedResourceVCores": 0, "numNonAMContainerPreempted": 0, "numAMContainerPreempted": 0, "resourceRequests": [ { "capability": { "memory": 1024, "virtualCores": 1 }, "nodeLabelExpression": "", "numContainers": 0, "priority": { "priority": 0 }, "relaxLocality": true, "resourceName": "*" } ], "logAggregationStatus": "NOT_START", "amNodeLabelExpression": "" }, { "id": "application_1461722876897_0002", "user": "admin", "name": "QuasiMonteCarlo", "queue": "default", "state": "FINISHED", "finalStatus": "SUCCEEDED", "progress": 100, "trackingUI": "History", "trackingUrl": "https://10-120-85-2:8090/proxy/application_1461722876897_0002/", "diagnostics": "Attempt recovered after RM restart", "clusterId": 1461743120947, "applicationType": "MAPREDUCE", "applicationTags": "", "startedTime": 1461741052993, "finishedTime": 1461741079483, "elapsedTime": 26490, "amContainerLogs": "https://10-120-85-2:8044/node/containerlogs/container_e11_1461722876897_0002_01_000001/admin", "amHostHttpAddress": "10-120-85-2:8044", "allocatedMB": -1, "allocatedVCores": -1, "runningContainers": -1, "memorySeconds": 158664, "vcoreSeconds": 52, "preemptedResourceMB": 0, "preemptedResourceVCores": 0, "numNonAMContainerPreempted": 0, "numAMContainerPreempted": 0, "amNodeLabelExpression": "" } ] } } - 结果分析：
- 命令：
- 获取Yarn资源的总体信息。
- 命令：
curl -k -i --negotiate -u : "https://10-120-85-102:8090/ws/v1/cluster/metrics"
- 运行结果：
{ "clusterMetrics": { "appsSubmitted": 2, "appsCompleted": 1, "appsPending": 0, "appsRunning": 1, "appsFailed": 0, "appsKilled": 0, "reservedMB": 0, "availableMB": 23552, "allocatedMB": 1024, "reservedVirtualCores": 0, "availableVirtualCores": 23, "allocatedVirtualCores": 1, "containersAllocated": 1, "containersReserved": 0, "containersPending": 0, "totalMB": 24576, "totalVirtualCores": 24, "totalNodes": 3, "lostNodes": 0, "unhealthyNodes": 0, "decommissionedNodes": 0, "rebootedNodes": 0, "activeNodes": 3, "rmMainQueueSize": 0, "schedulerQueueSize": 0, "stateStoreQueueSize": 0 } } - 结果分析：
- 命令：
父主题： YARN接口介绍
