# 使用Prometheus语句查询聚合指标 通过Prometheus语句可以将基础指标结合为聚合指标，方便快速查询并查看对象资源信息。本文档列出了以下聚合指标，涵盖CPU利用率、内存利用率、磁盘利用率、磁盘读写速度、网络传输错误率、网络丢包率、网络传输速率指标，支持Deployment、Pod、Container级别查询。本文档列出的聚合指标不代表仅支持以下聚合指标的查询，Prometheus语句语法详情可登录[Prometheus官网](https://prometheus.io/docs/prometheus/latest/querying/basics/)查看。 | 指标类别 | 指标名称 | 指标描述 | 指标实现 | |:---|:---|:---|:---| | CPU | deployment_cpu_usage | 查询负载的所有Pod的容器在不同的时间段使用的CPU总量占负载的所有Pod的容器的CPU Limit总量的比例 | ``` label_replace(sum by (created_by_name,namespace)((sum by (pod,namespace)(rate(container_cpu_usage_seconds_total{image!=""}[5m])) / sum by(pod,namespace)(container_spec_cpu_quota{image!=""} /100000)) *on(pod,namespace) group_left(created_by_kind,created_by_name) (count by (pod,created_by_kind,created_by_name,namespace)(kube_pod_info{created_by_kind='ReplicaSet'}))),"deployment","$1","created_by_name","(.*)-[^-]*")*100 ``` | | CPU | deployment_cpu_usage | 查询单个负载的所有Pod的容器在不同的时间段使用的CPU总量占负载的所有Pod的容器的CPU Limit总量的比例 | ``` label_replace(sum by (created_by_name,namespace)((sum by (pod,namespace)(rate(container_cpu_usage_seconds_total{image!=""}[5m])) / sum by(pod,namespace)(container_spec_cpu_quota{image!=""} /100000)) *on(pod,namespace) group_left(created_by_kind,created_by_name) (count by (pod,created_by_kind,created_by_name,namespace)(kube_pod_info{created_by_name =~ '{DEPLOYMENTNAME}.*',created_by_kind='ReplicaSet'}))),"deployment","$1","created_by_name","(.*)-[^-]*")*100 ``` | | CPU | pod_cpu_usage | 查询Pod的所有容器在不同的时间段CPU使用总量占Pod的所有容器CPU Limit总量的比例 | ``` sum(rate(container_cpu_usage_seconds_total{image != ""}[5m])) by (pod, namespace) / (sum(container_spec_cpu_quota{image != ""}/100000) by (pod, namespace)) * 100 ``` | | CPU | pod_cpu_usage | 查询单个Pod的所有容器在不同的时间段CPU使用总量占Pod的所有容器CPU Limit总量的比例 | ``` sum(rate(container_cpu_usage_seconds_total{image != "",pod="{PODNAME}"}[5m])) by (pod, namespace) / (sum(container_spec_cpu_quota{image != "",pod="{PODNAME}"}/100000) by (pod, namespace)) * 100 ``` | | CPU | container_cpu_usage | 查询容器在不同的时间段的CPU使用量占它们的CPU Limit量的比例 | ``` sum(rate(container_cpu_usage_seconds_total{image != ""}[5m])) by (pod,namespace,container) / (sum(container_spec_cpu_quota{image != ""}/100000) by (pod,namespace,container)) * 100 ``` | | CPU | container_cpu_usage | 查询单个容器在不同的时间段的CPU使用量占它们的CPU Limit量的比例 | ``` sum(rate(container_cpu_usage_seconds_total{image != "",pod="{PODNAME}",container="{CONTAINERNAME}"}[5m])) by (pod,namespace,container) / (sum(container_spec_cpu_quota{image != "",pod="{PODNAME}",container="{CONTAINERNAME}"}/100000) by (pod,namespace,container)) * 100 ``` | | 内存 | deployment_memory_usage | 查询负载的所有Pod的容器在不同的时间段使用的内存总量占负载的所有Pod的容器的内存Limit总量比例 | ``` label_replace(sum by (created_by_name,namespace)((sum by (pod,namespace)(container_memory_working_set_bytes{image!=""}) / sum by(pod,namespace)(container_spec_memory_limit_bytes{image!=""})) *on(pod,namespace) group_left(created_by_kind,created_by_name) (count by (pod,created_by_kind,created_by_name,namespace)(kube_pod_info{created_by_kind='ReplicaSet'}))),"deployment","$1","created_by_name","(.*)-[^-]*")*100 ``` | | 内存 | deployment_memory_usage | 查询单个负载的所有Pod的容器在不同的时间段使用的内存总量占负载的所有Pod的容器的内存Limit总量比例 | ``` label_replace(sum by (created_by_name,namespace)((sum by (pod,namespace)(container_memory_working_set_bytes{image!=""}) / sum by(pod,namespace)(container_spec_memory_limit_bytes{image!=""})) *on(pod,namespace) group_left(created_by_kind,created_by_name) (count by (pod,created_by_kind,created_by_name,namespace)(kube_pod_info{created_by_name =~ '{DEPLOYMENTNAME}.*',created_by_kind='ReplicaSet'}))),"deployment","$1","created_by_name","(.*)-[^-]*")*100 ``` | | 内存 | pod_memory_usage | 查询Pod的所有容器在不同的时间段内存使用总量占Pod的所有容器内存Limit总量 | ``` sum(container_memory_working_set_bytes{container!=""}) by (pod, namespace) / (sum(container_spec_memory_limit_bytes{container!=""}) by (pod, namespace)) * 100 ``` | | 内存 | pod_memory_usage | 查询单个Pod的所有容器在不同的时间段内存使用总量占Pod的所有容器内存Limit总量 | ``` sum(container_memory_working_set_bytes{container!="",pod="{PODNAME}"}) by (pod, namespace) / (sum(container_spec_memory_limit_bytes{container!="",pod="{PODNAME}"}) by (pod, namespace)) * 100 ``` | | 内存 | container_memory_usage | 查询容器在不同的时间段的内存使用量占它们的内存 Limit量的比例 | ``` sum(container_memory_working_set_bytes{container!=""}) by (pod, namespace,container) / (sum(container_spec_memory_limit_bytes{container!=""}) by (pod, namespace,container)) * 100 ``` | | 内存 | container_memory_usage | 查询单个容器在不同的时间段的内存使用量占它们的内存 Limit量的比例 | ``` sum(container_memory_working_set_bytes{container!="",pod="{PODNAME}",container="{CONTAINERNAME}"}) by (pod, namespace,container) / (sum(container_spec_memory_limit_bytes{container!="",pod="{PODNAME}",container="{CONTAINERNAME}"}) by (pod, namespace,container)) * 100 ``` | | 文件系统/磁盘 | deployment_disk_usage | 查询Deployment 磁盘利用率 | ``` label_replace(sum by (created_by_name,namespace)((sum by (pod,namespace)(container_fs_usage_bytes{image!=""}) / sum by(pod,namespace)(container_fs_limit_bytes{image!=""})) *on(pod,namespace) group_left(created_by_kind,created_by_name) (count by (pod,created_by_kind,created_by_name,namespace)(kube_pod_info{created_by_kind='ReplicaSet'}))),"deployment","$1","created_by_name","(.*)-[^-]*")*100 ``` | | 文件系统/磁盘 | deployment_disk_usage | 查询单个Deployment磁盘利用率 | ``` label_replace(sum by (created_by_name,namespace)((sum by (pod,namespace)(container_fs_usage_bytes{image!=""}) / sum by(pod,namespace)(container_fs_limit_bytes{image!=""})) *on(pod,namespace) group_left(created_by_kind,created_by_name) (count by (pod,created_by_kind,created_by_name,namespace)(kube_pod_info{created_by_name =~ '{DEPLOYMENTNAME}.*',created_by_kind='ReplicaSet'}))),"deployment","$1","created_by_name","(.*)-[^-]*")*100 ``` | | 文件系统/磁盘 | pod_disk_usage | 查询Pod 磁盘利用率 | ``` (sum(container_fs_usage_bytes{container!=""}) by (pod)) / (sum(container_fs_limit_bytes{container!=""}) by (pod)) * 100 ``` | | 文件系统/磁盘 | pod_disk_usage | 查询单个Pod磁盘利用率 | ``` (sum(container_fs_usage_bytes{container!="",pod="{PODNAME}"}) by (pod)) / (sum(container_fs_limit_bytes{container!="",pod="{PODNAME}"}) by (pod)) * 100 ``` | | 文件系统/磁盘 | container_disk_usage | 查询Container 磁盘利用率 | ``` (sum(container_fs_usage_bytes{container!=""}) by (pod,container)) / (sum(container_fs_limit_bytes{container!=""}) by (pod,container)) * 100 ``` | | 文件系统/磁盘 | container_disk_usage | 查询单个Container 磁盘利用率 | ``` (sum(container_fs_usage_bytes{container!="",pod="{PODNAME}",container="{CONTAINERNAME}"}) by (pod,container)) / (sum(container_fs_limit_bytes{container!="",pod="{PODNAME}",container="{CONTAINERNAME}"}) by (pod,container)) * 100 ``` | | 文件系统/磁盘 | disk_read_kilobytes | 查询磁盘的读取速率 | ``` sum(rate(container_fs_reads_total{container!=""}[5m])) by(namespace) ``` | | 文件系统/磁盘 | disk_write_kilobytes | 查询磁盘的写入速率 | ``` sum(rate(container_fs_writes_total{container!=""}[5m])) by(namespace) ``` | | 网络 | deployment_network_transmit_error_packets | 查询Deployment 传输错误率 | ``` label_replace(sum by (created_by_name,namespace)((sum by (pod,namespace)(container_network_transmit_packets_dropped_total{image!=""}) / sum by(pod,namespace)(container_network_transmit_packets_total{image!=""})) *on(pod,namespace) group_left(created_by_kind,created_by_name) (count by (pod,created_by_kind,created_by_name,namespace)(kube_pod_info{created_by_kind='ReplicaSet'}))),"deployment","$1","created_by_name","(.*)-[^-]*")*100 ``` | | 网络 | deployment_network_transmit_error_packets | 查询单个Deployment 传输错误率 | ``` label_replace(sum by (created_by_name,namespace)((sum by (pod,namespace)(container_network_transmit_packets_dropped_total{image!=""}) / sum by(pod,namespace)(container_network_transmit_packets_total{image!=""})) *on(pod,namespace) group_left(created_by_kind,created_by_name) (count by (pod,created_by_kind,created_by_name,namespace)(kube_pod_info{created_by_name =~ '{DEPLOYMENTNAME}.*',created_by_kind='ReplicaSet'}))),"deployment","$1","created_by_name","(.*)-[^-]*")*100 ``` | | 网络 | pod_network_transmit_error_packets | 查询Pod 传输错误率 | ``` sum(rate(container_network_transmit_packets_dropped_total{container!=""}[5m])) by(namespace,pod)/sum(rate(container_network_transmit_packets_total{container!=""}[5m])) by(namespace,pod)* 100 ``` | | 网络 | pod_network_transmit_error_packets | 查询单个Pod 传输错误率 | ``` sum(rate(container_network_transmit_packets_dropped_total{container!="",pod="{PODNAME}"}[5m])) by(namespace,pod)/sum(rate(container_network_transmit_packets_total{container!="",pod="{PODNAME}"}[5m])) by(namespace,pod)* 100 ``` | | 网络 | container_network_transmit_error_packets | 查询Container 传输错误率 | ``` sum(rate(container_network_transmit_packets_dropped_total{container!=""}[5m])) by(namespace,pod,container)/sum(rate(container_network_transmit_packets_total{container!=""}[5m])) by(namespace,pod,container)* 100 ``` | | 网络 | container_network_transmit_error_packets | 查询单个Container 传输错误率 | ``` sum(rate(container_network_transmit_packets_dropped_total{container!="",pod="{PODNAME}",container="{CONTAINERNAME}"}[5m])) by(namespace,pod,container)/sum(rate(container_network_transmit_packets_total{container!="",pod="{PODNAME}",container="{CONTAINERNAME}"}[5m])) by(namespace,pod,container)* 100 ``` | | 网络 | deployment_network_receive_error_packets | 查询Deployment 丢包率 | ``` label_replace(sum by (created_by_name,namespace)((sum by (pod,namespace)(container_network_receive_packets_dropped_total{image!=""}) / sum by(pod,namespace)(container_network_receive_packets_total{image!=""})) *on(pod,namespace) group_left(created_by_kind,created_by_name) (count by (pod,created_by_kind,created_by_name,namespace)(kube_pod_info{created_by_kind='ReplicaSet'}))),"deployment","$1","created_by_name","(.*)-[^-]*")*100 ``` | | 网络 | deployment_network_receive_error_packets | 查询单个Deployment 丢包率 | ``` label_replace(sum by (created_by_name,namespace)((sum by (pod,namespace)(container_network_receive_packets_dropped_total{image!=""}) / sum by(pod,namespace)(container_network_receive_packets_total{image!=""})) *on(pod,namespace) group_left(created_by_kind,created_by_name) (count by (pod,created_by_kind,created_by_name,namespace)(kube_pod_info{created_by_name =~ '{DEPLOYMENTNAME}.*',created_by_kind='ReplicaSet'}))),"deployment","$1","created_by_name","(.*)-[^-]*")*100 ``` | | 网络 | pod_network_receive_error_packets | 查询Pod 丢包率 | ``` sum(rate(container_network_receive_packets_dropped_total{container!=""}[5m])) by(namespace,pod)/sum(rate(container_network_receive_packets_total{container!=""}[5m])) by(namespace,pod)* 100 ``` | | 网络 | pod_network_receive_error_packets | 查询单个Pod 丢包率 | ``` sum(rate(container_network_receive_packets_dropped_total{container!="",pod="{PODNAME}"}[5m])) by(namespace,pod)/sum(rate(container_network_receive_packets_total{container!="",pod="{PODNAME}"}[5m])) by(namespace,pod)* 100 ``` | | 网络 | container_network_receive_error_packets | 查询Container 丢包率 | ``` sum(rate(container_network_receive_packets_dropped_total{container!=""}[5m])) by(namespace,pod,container)/sum(rate(container_network_receive_packets_total{container!=""}[5m])) by(namespace,pod,container)* 100 ``` | | 网络 | container_network_receive_error_packets | 查询单个Container 丢包率 | ``` sum(rate(container_network_receive_packets_dropped_total{container!="",pod="{PODNAME}",container="{CONTAINERNAME}"}[5m])) by(namespace,pod,container)/sum(rate(container_network_receive_packets_total{container!="",pod="{PODNAME}",container="{CONTAINERNAME}"}[5m])) by(namespace,pod,container)* 100 ``` | | 网络 | network_transmit_bytes | 查询网络发送速率 | ``` sum(rate(container_network_transmit_bytes_total{container!=""}[5m])) by(namespace) ``` | | 网络 | network_receive_bytes | 查询网络接收速率 | ``` sum(rate(container_network_receive_bytes_total{container!=""}[5m])) by(namespace) ``` | ![](https://support.huaweicloud.com/usermanual-cci2/public_sys-resources/caution_3.0-zh-cn.png) - 查询单个Deployment对象资源时，将语句中{DEPLOYMENTNAME}修改为待查询的Deployment名。 - 查询单个Pod对象资源时，将语句中{PODNAME}修改为待查询的Pod名。 - 查询单个Container对象资源时，将语句中{PODNAME}修改为待查询Container所属的Pod名，将{CONTAINERNAME}改为待查询的Container名。 - 当前CPU和内存聚合监控指标（如使用率）的计算依赖于容器的Limit配置。当存在容器Limit配置为0时，CCI提供的Prometheus语句查询出的聚合监控指标数据将会失真。为保障监控数据的有效性，建议您为各个容器合理配置Limit值。