Compute
Elastic Cloud Server
Huawei Cloud Flexus
Bare Metal Server
Auto Scaling
Image Management Service
Dedicated Host
FunctionGraph
Cloud Phone Host
Huawei Cloud EulerOS
Networking
Virtual Private Cloud
Elastic IP
Elastic Load Balance
NAT Gateway
Direct Connect
Virtual Private Network
VPC Endpoint
Cloud Connect
Enterprise Router
Enterprise Switch
Global Accelerator
Management & Governance
Cloud Eye
Identity and Access Management
Cloud Trace Service
Resource Formation Service
Tag Management Service
Log Tank Service
Config
OneAccess
Resource Access Manager
Simple Message Notification
Application Performance Management
Application Operations Management
Organizations
Optimization Advisor
IAM Identity Center
Cloud Operations Center
Resource Governance Center
Migration
Server Migration Service
Object Storage Migration Service
Cloud Data Migration
Migration Center
Cloud Ecosystem
KooGallery
Partner Center
User Support
My Account
Billing Center
Cost Center
Resource Center
Enterprise Management
Service Tickets
HUAWEI CLOUD (International) FAQs
ICP Filing
Support Plans
My Credentials
Customer Operation Capabilities
Partner Support Plans
Professional Services
Analytics
MapReduce Service
Data Lake Insight
CloudTable Service
Cloud Search Service
Data Lake Visualization
Data Ingestion Service
GaussDB(DWS)
DataArts Studio
Data Lake Factory
DataArts Lake Formation
IoT
IoT Device Access
Others
Product Pricing Details
System Permissions
Console Quick Start
Common FAQs
Instructions for Associating with a HUAWEI CLOUD Partner
Message Center
Security & Compliance
Security Technologies and Applications
Web Application Firewall
Host Security Service
Cloud Firewall
SecMaster
Anti-DDoS Service
Data Encryption Workshop
Database Security Service
Cloud Bastion Host
Data Security Center
Cloud Certificate Manager
Edge Security
Situation Awareness
Managed Threat Detection
Blockchain
Blockchain Service
Web3 Node Engine Service
Media Services
Media Processing Center
Video On Demand
Live
SparkRTC
MetaStudio
Storage
Object Storage Service
Elastic Volume Service
Cloud Backup and Recovery
Storage Disaster Recovery Service
Scalable File Service Turbo
Scalable File Service
Volume Backup Service
Cloud Server Backup Service
Data Express Service
Dedicated Distributed Storage Service
Containers
Cloud Container Engine
SoftWare Repository for Container
Application Service Mesh
Ubiquitous Cloud Native Service
Cloud Container Instance
Databases
Relational Database Service
Document Database Service
Data Admin Service
Data Replication Service
GeminiDB
GaussDB
Distributed Database Middleware
Database and Application Migration UGO
TaurusDB
Middleware
Distributed Cache Service
API Gateway
Distributed Message Service for Kafka
Distributed Message Service for RabbitMQ
Distributed Message Service for RocketMQ
Cloud Service Engine
Multi-Site High Availability Service
EventGrid
Dedicated Cloud
Dedicated Computing Cluster
Business Applications
Workspace
ROMA Connect
Message & SMS
Domain Name Service
Edge Data Center Management
Meeting
AI
Face Recognition Service
Graph Engine Service
Content Moderation
Image Recognition
Optical Character Recognition
ModelArts
ImageSearch
Conversational Bot Service
Speech Interaction Service
Huawei HiLens
Video Intelligent Analysis Service
Developer Tools
SDK Developer Guide
API Request Signing Guide
Terraform
Koo Command Line Interface
Content Delivery & Edge Computing
Content Delivery Network
Intelligent EdgeFabric
CloudPond
Intelligent EdgeCloud
Solutions
SAP Cloud
High Performance Computing
Developer Services
ServiceStage
CodeArts
CodeArts PerfTest
CodeArts Req
CodeArts Pipeline
CodeArts Build
CodeArts Deploy
CodeArts Artifact
CodeArts TestPlan
CodeArts Check
CodeArts Repo
Cloud Application Engine
MacroVerse aPaaS
KooMessage
KooPhone
KooDrive

Monitoring Metrics for Elasticsearch Clusters in Cloud Eye

Updated on 2025-01-23 GMT+08:00

You can use Cloud Eye to monitor cluster metrics of CSS in real time and quickly handle exceptions.

Function

You can use Cloud Eye to monitor cluster metrics of CSS in real time and quickly handle exceptions. This topic describes CSS cluster and node metrics that can be monitored through Cloud Eye, as well as their namespaces and dimensions. You can use the management console or APIs provided by Cloud Eye to check the monitoring metrics and alarms generated for CSS.

Namespace

SYS.ES

Cluster Monitoring Metrics

  • Table 1 describes the monitoring metrics for CSS clusters.
  • Monitored objects: CSS clusters. (This document describes the monitoring metrics for Elasticsearch and OpenSearch clusters.) The dimension name is cluster_id.
  • Monitoring period (original metrics): 1 minute.
NOTE:

Accumulated value: The value is accumulated from when a node is started. After the node is restarted, the value is reset to zero and starts accumulating again.

Table 1 CSS cluster metrics

Metric ID

Metric

Description

Value Range

status

Cluster Health Status

Health status of the monitored object

0,1,2,3

  • 0: The cluster is 100% available.
  • 1: All primary shards are allocated, but at least one replica is missing. No data is missing, so search results will still be complete. However, high availability is compromised to some degree. If more shards disappear, you might lose data. Think of this status as a warning that should prompt investigation.
  • 2: Data is missing and the cluster fails to work.
  • 3: Failed to obtain the cluster status.

disk_util

Disk Usage

Disk usage of the monitored object.

Unit: %

0-100%

max_jvm_heap_usage

Max. JVM Heap Usage

Maximum JVM heap usage of nodes in a CSS cluster.

Unit: %

0-100%

max_jvm_young_gc_time

Max. JVM Young GC Duration

Maximum accumulated JVM Young GC duration of nodes in a CSS cluster.

Unit: ms

≥ 0 ms

max_jvm_young_gc_count

Max. JVM Young GC Count

Maximum accumulated JVM Young GC count of nodes in a CSS cluster.

≥ 0

max_jvm_old_gc_time

Max. JVM Old GC Duration

Maximum accumulated JVM Old GC duration of nodes in a CSS cluster.

Unit: ms

≥ 0 ms

max_jvm_old_gc_count

Max. JVM Old GC Count

Maximum accumulated JVM Old GC count of nodes in a CSS cluster.

≥ 0

total_fs_size

Total Size of File Systems

Total size of file systems in a CSS cluster.

Unit: byte

≥ 0 bytes

free_fs_size

Available Size of File Systems

Available size of file systems in a CSS cluster.

Unit: byte

≥ 0 bytes

max_cpu_usage

Max. CPU Usage

Maximum node CPU usage in a CSS cluster.

Unit: %

0-100%

max_cpu_time_of_jvm_process

Max. CPU Time of JVM Process

Maximum accumulated CPU usage duration of node JVM processes in a CSS cluster.

Unit: ms

≥ 0 ms

max_virtual_memory_size_of_jvm_process

Max. Virtual Memory Size of JVM Process

Maximum virtual memory size of node JVM processes in a CSS cluster.

Unit: byte

≥ 0 bytes

max_current_opened_http_count

Current Max. Opened HTTP Connections

Maximum number of HTTP connections that are currently open for nodes in a CSS cluster.

≥ 0

max_total_opened_http_count

Total Max. Opened HTTP Connections

Maximum number of HTTP connections that were open for nodes in a CSS cluster.

≥ 0

indices_count

Indexes

Number of indexes in a CSS cluster

≥ 0

total_shards_count

Shards

Number of shards in a CSS cluster

≥ 0

primary_shards_count

Primary Shards

Number of primary shards in a CSS cluster

≥ 0

docs_count

Documents

Number of documents in a CSS cluster

≥ 0

docs_deleted_count

Deleted Documents

Number of documents deleted in a CSS cluster

≥ 0

nodes_count

Nodes

Number of nodes in a CSS cluster

≥ 0

data_nodes_count

Data Nodes

Number of data nodes in a CSS cluster

≥ 0

coordinating_nodes_count

Coordinator Nodes

Number of coordinator nodes in a CSS cluster

≥ 0

master_nodes_count

Master Nodes

Number of master nodes in a CSS cluster

≥ 0

ingest_nodes_count

Client Nodes

Number of client nodes in a CSS cluster

≥ 0

max_load_average

Max. Node Load

Maximum number of average queued tasks per minute on nodes in a cluster

≥ 0

avg_cpu_usage

Avg. CPU Usage

Average node CPU usage in a CSS cluster.

Unit: %

0-100%

avg_load_average

Avg. Node Load

Average number of queued tasks per minute on nodes in a CSS cluster

≥ 0

avg_jvm_heap_usage

Avg. JVM Heap Usage

Average node JVM heap usage in a CSS cluster.

Unit: %

0-100%

max_open_file_descriptors

Max. Open File Descriptors

Maximum number of node file descriptors that are currently open in a CSS cluster.

≥ 0

avg_open_file_descriptors

Avg. Open File Descriptors

Average number of node file descriptors that are currently open in a CSS cluster.

≥ 0

sum_max_file_descriptors

Max. Allowed File Descriptors

Maximum number of allowed node file descriptors in a CSS cluster.

≥ 0

sum_open_file_descriptors

Open File Descriptors

Number of node file descriptors that are currently open in a cluster.

≥ 0

sum_thread_pool_write_queue

Tasks in Write Queue

Number of tasks in queue for the write thread pool.

≥ 0

sum_thread_pool_search_queue

Tasks in Search Queue

Total number of tasks in queue for the search thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_force_merge_queue

Tasks in ForceMerge Queue

Total number of tasks in queue for the force merge thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_write_rejected

Rejected Tasks in Write Queue

Total number of tasks rejected by the write thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_search_rejected

Rejected Tasks in Search Queue

Total number of tasks rejected by the search thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_force_merge_rejected

Rejected Tasks in ForceMerge Queue

Total number of tasks rejected by the force merge thread pool on nodes in a CSS cluster.

≥ 0

max_thread_pool_search_queue

Max. Tasks in Search Queue

Maximum number of tasks in queue for the search thread pool on nodes in a CSS cluster.

≥ 0

max_thread_pool_force_merge_queue

Max. Tasks in ForceMerge Queue

Maximum number of tasks in queue for the force merge thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_write_threads

Size of Write Thread Pool

Total size of the write thread pools on nodes in the CSS cluster.

≥ 0

sum_thread_pool_search_threads

Size of Search Thread Pool

Total size of the search thread pools on nodes in the CSS cluster.

≥ 0

sum_thread_pool_force_merge_threads

Size of ForceMerge Thread Pool

Total size of the force merge thread pools on nodes in the CSS cluster.

≥ 0

avg_thread_pool_write_queue

Avg. Tasks in Write Queue

Average number of tasks in queue for the write thread pool on nodes in a CSS cluster.

≥ 0

avg_thread_pool_search_queue

Avg. Tasks in Search Queue

Average number of tasks in queue for the search thread pool on nodes in a CSS cluster.

≥ 0

avg_thread_pool_force_merge_queue

Avg. Tasks in ForceMerge Queue

Average number of tasks in queue for the force merge thread pool on nodes in a CSS cluster.

≥ 0

avg_thread_pool_search_threads

Avg. Size of Search Thread Pool

Average size of the search thread pool on nodes in a CSS cluster.

≥ 0

avg_thread_pool_write_threads

Avg. Size of Write Thread Pool

Average size of the write thread pool on nodes in a CSS cluster.

≥ 0

avg_thread_pool_force_merge_threads

Avg. Size of ForceMerge Thread Pool

Average size of the force merge thread pool on nodes in a CSS cluster.

≥ 0

avg_thread_pool_write_rejected

Avg. Rejected Tasks in Write Queue

Average number of tasks rejected by the write thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_flush_queue

Tasks in Flush Queue

Total number of tasks in queue for the flush thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_flush_rejected

Rejected Tasks in Flush Queue

Total number of tasks rejected by the flush thread pool on nodes in a CSS cluster.

≥ 0

max_thread_pool_flush_queue

Max. Tasks in Flush Queue

Maximum number of tasks in queue for the flush thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_flush_threads

Size of Flush Thread Pool

Total size of the flush thread pools on nodes in a CSS cluster.

≥ 0

avg_thread_pool_flush_queue

Avg. Tasks in Flush Queue

Average number of tasks in queue for the flush thread pool on nodes in a CSS cluster.

≥ 0

avg_thread_pool_flush_threads

Avg. Size of Flush Thread Pool

Average number of tasks in queue for the flush thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_generic_queue

Tasks in Generic Queue

Total number of tasks in queue for the generic thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_generic_rejecte

Rejected Tasks in Generic Queue

Total number of tasks rejected by the generic thread pool on nodes in a CSS cluster.

≥ 0

max_thread_pool_generic_queue

Max. Tasks in Generic Queue

Maximum number of tasks in queue for the generic thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_generic_threads

Size of Generic Thread Pool

Total size of the generic thread pools on nodes in a CSS cluster.

≥ 0

avg_thread_pool_generic_queue

Avg. Tasks in Generic Queue

Average number of tasks in queue for the generic thread pool on nodes in a CSS cluster.

≥ 0

avg_thread_pool_generic_threads

Avg. Size of Generic Thread Pool

Average number of tasks in queue for the generic thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_management_queue

Tasks in Management Queue

Total number of tasks in queue for the management thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_management_rejected

Rejected Tasks in Management Queue

Total number of tasks rejected by the management thread pool on nodes in a CSS cluster.

≥ 0

max_thread_pool_management_queue

Max. Tasks in Management Queue

Maximum number of tasks in queue for the management thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_management_threads

Size of Management Thread Pool

Total size of the management thread pools on nodes in a CSS cluster.

≥ 0

avg_thread_pool_management_queue

Avg. Tasks in Management Queue

Average number of tasks in queue for the management thread pool on nodes in a CSS cluster.

≥ 0

avg_thread_pool_management_threads

Avg. Size of Management Thread Pool

Average number of tasks in queue for the management thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_refresh_queue

Tasks in Refresh Queue

Total number of tasks in queue for the refresh thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_refresh_rejected

Rejected Tasks in Refresh Queue

Total number of tasks rejected by the refresh thread pool on nodes in a CSS cluster.

≥ 0

max_thread_pool_refresh_queue

Max. Tasks in Refresh Queue

Maximum number of tasks in queue for the refresh thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_refresh_threads

Size of Refresh Thread Pool

Total size of the refresh thread pools on nodes in a CSS cluster.

≥ 0

avg_thread_pool_refresh_queue

Avg. Tasks in Refresh Queue

Average number of tasks in queue for the refresh thread pool on nodes in a CSS cluster.

≥ 0

avg_thread_pool_refresh_threads

Avg. Size of Refresh Thread Pool

Average number of tasks in queue for the refresh thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_obs_searcher_queue

Tasks in OBS Searcher Queue

Total number of tasks in queue for the OBS Searcher thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_obs_searcher_rejected

Rejected Tasks in OBS Searcher Queue

Total number of tasks rejected by the OBS Searcher thread pool on nodes in a CSS cluster.

≥ 0

max_thread_pool_obs_searcher_queue

Max. Tasks in OBS Searcher Queue

Maximum number of tasks in queue for the OBS Searcher thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_obs_searcher_threads

Size of OBS Searcher Thread Pool

Total size of the OBS Searcher thread pools on nodes in a CSS cluster.

≥ 0

avg_thread_pool_obs_searcher_queue

Avg. Tasks in OBS Searcher Queue

Average number of tasks in queue for the OBS Searcher thread pool on nodes in a CSS cluster.

≥ 0

avg_thread_pool_obs_searcher_threads

Avg. Size of OBS Searcher Thread Pool

Average number of tasks in queue for the OBS Searcher thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_obs_queue

Tasks in OBS Queue

Total number of tasks in queue for the OBS thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_obs_rejected

Rejected Tasks in OBS Queue

Total number of tasks rejected by the OBS thread pool on nodes in a CSS cluster.

≥ 0

max_thread_pool_obs_queue

Max. Tasks in OBS Queue

Maximum number of tasks in queue for the OBS thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_obs_threads

Size of OBS Thread Pool

Total size of the OBS thread pools on nodes in a CSS cluster.

≥ 0

avg_thread_pool_obs_queue

Avg. Tasks in OBS Queue

Average number of tasks in queue for the OBS thread pool on nodes in a CSS cluster.

≥ 0

avg_thread_pool_obs_threads

Avg. Size of OBS Thread Pool

Average number of tasks in queue for the OBS thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_obs_upload_queue

Tasks in OBS Upload Queue

Total number of tasks in queue for the OBS Upload thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_obs_upload_rejected

Rejected Tasks in OBS Upload Queue

Total number of tasks rejected by the OBS Upload thread pool on nodes in a CSS cluster.

≥ 0

max_thread_pool_obs_upload_queue

Max. Tasks in OBS Upload Queue

Maximum number of tasks in queue for the OBS Upload thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_obs_upload_threads

Size of OBS Upload Thread Pool

Total size of the OBS Upload thread pools on nodes in a CSS cluster.

≥ 0

avg_thread_pool_obs_upload_queue

Avg. Tasks in OBS Upload Queue

Average number of tasks in queue for the OBS Upload thread pool on nodes in a CSS cluster.

≥ 0

avg_thread_pool_obs_upload_threads

Avg. Size of OBS Upload Thread Pool

Average number of tasks in queue for the OBS Upload thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_obs_download_queue

Tasks in OBS Download Queue

Total number of tasks in queue for the OBS Download thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_obs_download_rejected

Rejected Tasks in OBS Download Queue

Total number of tasks rejected by the OBS Download thread pool on nodes in a CSS cluster.

≥ 0

max_thread_pool_obs_download_queue

Max. Tasks in OBS Download Queue

Maximum number of tasks in queue for the OBS Download thread pool on nodes in a CSS cluster.

≥ 0

sum_thread_pool_obs_download_threads

Size of OBS Download Thread Pool

Total size of the OBS Download thread pools on nodes in a CSS cluster.

≥ 0

avg_thread_pool_obs_download_queue

Avg. Tasks in OBS Download Queue

Average number of tasks in queue for the OBS Download thread pool on nodes in a CSS cluster.

≥ 0

avg_thread_pool_obs_download_threads

Avg. Size of OBS Download Thread Pool

Average number of tasks in queue for the OBS Download thread pool on nodes in a CSS cluster.

≥ 0

min_free_fs_size

Min. Available Storage Space

Minimum available storage space of nodes in a CSS cluster.

Unit: byte

≥ 0 bytes

avg_jvm_old_gc_count

Avg. GCs of Old-Generation JVM

Average number of old-generation garbage collections of nodes in a CSS cluster.

≥ 0

avg_jvm_old_gc_time

Avg. GC Duration of Old-Generation JVM

Average old-generation garbage collection duration of nodes in a CSS cluster.

Unit: ms

≥ 0 ms

avg_jvm_young_gc_count

Avg. GCs of Young-Generation JVM

Average number of young-generation garbage collections of nodes in a CSS cluster.

≥ 0

avg_jvm_young_gc_time

Avg. GC Duration of Young-Generation JVM

Average young-generation garbage collection duration of nodes in a CSS cluster.

Unit: ms

≥ 0 ms

avg_max_file_descriptors

Avg. Maximum Allowed File Descriptors

Average value of the maximum number of allowed file descriptors on each node in a CSS cluster.

≥ 0

avg_mem_free_in_bytes

Avg. Available Memory

Average unused memory capacity of nodes in a CSS cluster.

Unit: byte

≥ 0 bytes

avg_mem_free_percent

Avg. Available Memory Percentage

Average percentage of unused memory of nodes in a CSS cluster.

Unit: %

0-100%

avg_mem_used_in_bytes

Avg. Used Memory

Average used memory of nodes in a CSS cluster.

Unit: byte

≥ 0 bytes

avg_mem_used_percent

Avg. Used Memory Percentage

Average percentage of used memory of nodes in a CSS cluster.

Unit: %

0-100%

max_mem_free_in_bytes

Max. Available Memory

Maximum unused memory of nodes in a CSS cluster.

Unit: byte

≥ 0 bytes

max_mem_free_percent

Max. Available Memory Percentage

Maximum percentage of unused memory of nodes in a CSS cluster.

Unit: %

0-100%

max_mem_used_in_bytes

Max. Used Memory

Maximum used memory of nodes in a CSS cluster.

Unit: byte

≥ 0 bytes

max_mem_used_percent

Max. Used Memory Percentage

Maximum percentage of used memory of nodes in a CSS cluster.

Unit: %

0-100%

sum_jvm_old_gc_count

Total GCs of Old-Generation JVM

Number of old-generation garbage collections of nodes in a CSS cluster.

≥ 0

sum_jvm_old_gc_time

Total GC Duration of Old-Generation JVM

Total old-generation garbage collection duration of nodes in the CSS cluster.

Unit: ms

≥ 0ms

sum_jvm_young_gc_count

Total GCs of Young-Generation JVM

Number of young-generation garbage collections of nodes in a CSS cluster.

≥ 0

sum_jvm_young_gc_time

Total GC Duration of Young-Generation JVM

Total young-generation garbage collection duration of nodes in the CSS cluster.

Unit: ms

≥ 0 ms

sum_current_opened_http_count

Currently Open HTTP Connections

Number of HTTP connections that are open on nodes in a CSS cluster.

≥ 0

sum_total_opened_http_count

Historical Open HTTP Connections

Number of HTTP connections that were open on nodes in a CSS cluster.

≥ 0

IndexingLatency

Average Index Latency

Average time required for a shard to complete an indexing operation.

Unit: ms

≥ 0 ms

IndexingRate

Average Index Rate

Average number of index operations per second in a cluster.

≥ 0

SearchLatency

Average Search Latency

Average time required for a shard to complete a search operation.

Unit: ms

≥ 0 ms

SearchRate

Average QPS

Average queries per second (QPS) in a cluster.

≥ 0

task_max_running_time

Max. Task Runtime

Duration of the most time-consuming search or write task in the cluster.

≥ 0 ms

number_of_pending_tasks

Number of Pending Tasks in Queue

Number of pending tasks to be processed by the master node in the CSS cluster.

≥ 0

sum_disk_read_requests_rate

Total Disk Read IOPS

Number of disk read requests to all nodes in the CSS cluster per second.

≥ 0

sum_disk_write_requests_rate

Total Disk Write IOPS

Number of disk write requests to all nodes in the CSS cluster per second.

≥ 0

sum_disk_read_bytes_rate

Total Disk Read Rate

Number of bytes of data read from the disks of all nodes in the CSS cluster per second.

≥ 0

sum_disk_write_bytes_rate

Total Disk Write Rate

Number of bytes of data written into the disks of all nodes in the CSS cluster per second.

≥ 0

avg_vector_index_off_heap_used_in_bytes

Average Off-Heap Memory Usage of Vector Indexes

Average off-heap memory (bytes) used for querying vector indexes on each node in a CSS cluster.

≥ 0 bytes

avg_vector_index_off_heap_usage

Average Off-Heap Memory Utilization (%) of Vector Indexes

Average off-heap memory (in %) used for querying vector indexes on each node in a CSS cluster.

This metric is calculated only when vector index queries occur.

0-100%

max_vector_index_off_heap_used_in_bytes

Maximum Off-Heap Memory Usage of Vector Indexes

Maximum off-heap memory (bytes) used for querying vector indexes on each node in a CSS cluster.

≥ 0 bytes

max_vector_index_off_heap_usage

Maximum Off-Heap Memory Utilization (%) of Vector Indexes

Maximum off-heap memory (in %) used for querying vector indexes on each node in a CSS cluster.

This metric is calculated only when vector index queries occur.

0-100%

vector_index_circuit_breaker_status

Vector Index Circuit Breaker Status

Circuit breaker status for vector indexes.

  • 0: Not enabled.
  • 1: Enabled. Check whether the cluster's off-heap memory usage exceeds the threshold.

0, 1

request_count

Total Requests

Total number of requests sent to all nodes in a CSS cluster.

≥ 0

successfully_request_count

Successful requests

Number of requests successfully handled by all nodes in a CSS cluster.

Status code: 200.

≥ 0

failed_request_count

Failed requests

Number of failed requests for all nodes in a CSS cluster.

Status code: non-200.

≥ 0

limited_request_count

Throttled requests

Number of throttled requests for all nodes in a CSS cluster.

Status code: 429.

≥ 0

cold_data_storage

The size of cold data stored

The total size of cold data stored in a CSS cluster, which is the total size of the data of frozen indexes stored in an OBS bucket. Users cannot check this OBS bucket on the OBS console.

≥ 0 bytes

number_of_index_creation_failures

Number of index creation failures.

Number of index creation failures in a CSS cluster.

≥ 0

shard_doc_exceed_threshold_count

Number of shards where the number of documents exceeds the threshold.

Number of shards where the number of documents exceeds the threshold in a CSS cluster.

≥ 0

Node Monitoring Metrics

  • Table 2 describes the monitoring metrics for CSS cluster nodes.
  • Monitored object: CSS cluster nodes. The dimension name is cluster_id,instance_id.
  • Monitoring period (original metrics): 1 minute
NOTE:

Accumulated value: The value is accumulated from when a node is started. After the node is restarted, the value is reset to zero and starts accumulating again.

Table 2 CSS node monitoring metrics

Metric ID

Metric

Description

Value Range

jvm_heap_usage

JVM Heap Usage

JVM heap memory usage of a node.

Unit: %

0-100%

cpu_usage

CPU Usage

CPU usage.

Unit: %

0-100%

disk_util

Disk Usage

Memory usage of a monitored object.

Unit: %

0-100%

load_average

Average Load

Average number of queued tasks per minute on a node

≥ 0

open_file_descriptors

Open File Descriptors

Number of opened file descriptors on a node

≥ 0

max_file_descriptors

Max. Allowed File Descriptors

Maximum number of allowed file descriptors

≥ 0

thread_pool_write_queue

Tasks in Write Queue

Number of tasks in queue for the write thread pool.

≥ 0

thread_pool_search_queue

Tasks in Search Queue

Number of tasks in queue for the search thread pool.

≥ 0

thread_pool_force_merge_queue

Tasks in ForceMerge Queue

Number of tasks in queue for the force merge thread pool.

≥ 0

thread_pool_write_rejected

Rejected Tasks in Write Queue

Number of tasks rejected by the write thread pool.

≥ 0

thread_pool_search_rejected

Rejected Tasks in Search Queue

Number of tasks rejected by the search thread pool.

≥ 0

thread_pool_force_merge_rejected

Rejected Tasks in ForceMerge Queue

Number of tasks rejected by the force merge thread pool

≥ 0

thread_pool_write_threads

Size of Write Thread Pool

Size of a write thread pool

≥ 0

thread_pool_search_threads

Size of Search Thread Pool

Size of a search thread pool

≥ 0

thread_pool_force_merge_threads

Size of ForceMerge Thread Pool

Size of a force merge thread pool

≥ 0

thread_pool_flush_queue

Tasks in Flush Queue

Number of tasks in queue for the flush thread pool.

≥ 0

thread_pool_flush_rejected

Rejected Tasks in Flush Queue

Number of tasks rejected by the flush thread pool.

≥ 0

thread_pool_flush_threads

Size of Flush Thread Pool

Size of the flush thread pool.

≥ 0

thread_pool_generic_queue

Tasks in Generic Queue

Number of tasks in queue for the generic thread pool.

≥ 0

thread_pool_generic_rejected

Rejected Tasks in Generic Queue

Number of tasks rejected by the generic thread pool.

≥ 0

thread_pool_generic_threads

Size of Generic Thread Pool

Size of the generic thread pool.

≥ 0

thread_pool_management_queue

Tasks in Management Queue

Number of tasks in queue for the management thread pool.

≥ 0

thread_pool_management_rejected

Rejected Tasks in Management Queue

Number of tasks rejected by the management thread pool.

≥ 0

thread_pool_management_threads

Size of Management Thread Pool

Size of the management thread pool.

≥ 0

thread_pool_refresh_queue

Tasks in Refresh Queue

Number of tasks in queue for the refresh thread pool.

≥ 0

thread_pool_refresh_rejected

Rejected Tasks in Refresh Queue

Number of tasks rejected by the refresh thread pool.

≥ 0

thread_pool_refresh_threads

Size of Refresh Thread Pool

Size of the refresh thread pool.

≥ 0

thread_pool_obs_searcher_queue

Tasks in OBS Searcher Queue

Number of tasks in queue for the OBS Searcher thread pool.

≥ 0

thread_pool_obs_searcher_rejected

Rejected Tasks in OBS Searcher Queue

Number of tasks rejected by the OBS Searcher thread pool.

≥ 0

thread_pool_obs_searcher_threads

Size of OBS Searcher Thread Pool

Size of the OBS Searcher thread pool.

≥ 0

thread_pool_obs_queue

Tasks in OBS Queue

Number of tasks in queue for the OBS thread pool.

≥ 0

thread_pool_obs_rejected

Rejected Tasks in OBS Queue

Number of tasks rejected by the OBS thread pool.

≥ 0

thread_pool_obs_threads

Size of OBS Thread Pool

Size of the OBS thread pool.

≥ 0

thread_pool_obs_upload_queue

Tasks in OBS Upload Queue

Number of tasks in queue for the OBS Upload thread pool.

≥ 0

thread_pool_obs_upload_rejected

Rejected Tasks in OBS Upload Queue

Number of tasks rejected by the OBS Upload thread pool.

≥ 0

thread_pool_obs_upload_threads

Size of OBS Upload Thread Pool

Size of the OBS Upload thread pool.

≥ 0

thread_pool_obs_download_queue

Tasks in OBS Download Queue

Number of tasks in queue for the OBS Download thread pool.

≥ 0

thread_pool_obs_download_rejected

Rejected Tasks in OBS Download Queue

Number of tasks rejected by the OBS Upload thread pool.

≥ 0

thread_pool_obs_download_threads

Size of OBS Download Thread Pool

Size of the OBS Download thread pool.

≥ 0

free_fs_size

Available Size of File Systems

Available size of file systems in a CSS cluster.

Unit: byte

≥ 0 bytes

total_fs_size

Total Size of File Systems

Total size of file systems in a CSS cluster.

Unit: byte

≥ 0 bytes

jvm_old_gc_count

Total GCs of Old-Generation JVM

Number of old-generation garbage collection times

≥ 0

jvm_old_gc_time

Total GC Duration of Old-Generation JVM

Time spent on old-generation garbage collection

Unit: ms

≥ 0 ms

jvm_young_gc_count

Total GCs of Young-Generation JVM

Number of young-generation garbage collection times

≥ 0

jvm_young_gc_time

GC Duration of Young-Generation JVM

Time spent on young-generation garbage collection

Unit: ms

≥ 0 ms

mem_free_in_bytes

Available Memory

Unused memory space of a node.

Unit: byte

≥ 0 bytes

mem_free_percent

Available Memory Percentage

Percentage of unused memory space on a node.

≥ 0

mem_used_in_bytes

Used Memory

Used memory space of a node.

Unit: byte

≥ 0 bytes

current_opened_http_count

Currently Open HTTP Connections

Number of HTTP connections currently open on a node

≥ 0

total_opened_http_count

Total Open HTTP Connections

Total number of HTTP connections on a node

≥ 0

disk_read_requests_rate

Disk Read IOPS

Number of disk read requests to a node in the CSS cluster per second.

≥ 0

disk_write_requests_rate

Disk Write IOPS

Number of disk write requests to a node in the CSS cluster per second.

≥ 0

disk_read_bytes_rate

Disk Read Rate

Number of bytes of data read from the disks of a node in the CSS cluster per second.

≥ 0

disk_write_bytes_rate

Disk Write Rate

Number of bytes of data written into the disks of a node in the CSS cluster per second.

≥ 0

shards_count

Shards

Number of shards on a node in the CSS cluster.

≥ 0

vector_index_off_heap_used_in_bytes

Off-Heap Memory Usage of Vector Indexes

Off-heap memory (bytes) used for querying vector indexes on nodes in a CSS cluster.

≥ 0 bytes

vector_index_off_heap_usage

Off-Heap Memory Utilization (%) of Vector Indexes

Off-heap memory (in %) used for querying vector indexes on nodes in a CSS cluster.

0-100%

Dimension

Table 3 Dimension description

Key

Value

cluster_id

CSS cluster

cluster_id,instance_id

CSS cluster node

We use cookies to improve our site and your experience. By continuing to browse our site you accept our cookie policy. Find out more

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback