Updated on 2024-12-02 GMT+08:00

Monitoring Metrics of Clusters

Function

This topic describes CSS metrics that can be monitored by Cloud Eye as well as their namespaces and dimensions.

Namespace

SYS.ES

Monitoring Metrics

  • Table 1 describes the monitoring metrics of CSS clusters.
  • Monitored object: CSS cluster
  • Monitoring period (original metric): 1 minute

Accumulated value: The value is accumulated from the time when a node is started. After the node is restarted, the value is reset to zero and accumulated again.

Table 1 CSS metrics

Metric ID

Metric

Description

Value Range

Monitored Object

Monitoring Interval (Raw Data)

status

Cluster Health Status

Health status of the monitored object

0,1,2,3

0: The cluster is 100% available.

1: The data is complete while some replicas are missing. Exceptions may occur because the high availability is compromised. This is a warning that should prompt investigation.

2: Data is missing and the cluster fails to work.

3: The cluster status is not obtained.

CSS cluster

1 minute

disk_util

Disk Usage

Disk usage of the monitored object.

Unit: %

0-100%

CSS cluster

1 minute

max_jvm_heap_usage

Max. JVM Heap Usage

Maximum JVM heap usage of nodes in a CSS cluster.

Unit: %

0-100%

CSS cluster

1 minute

max_jvm_young_gc_time

Max. JVM Young GC Duration

Maximum accumulated JVM Young GC duration of nodes in a CSS cluster.

Unit: ms

≥ 0 ms

CSS cluster

1 minute

max_jvm_young_gc_count

Max. JVM Young GC Count

Maximum accumulated JVM Young GC count of nodes in a CSS cluster.

≥ 0

CSS cluster

1 minute

max_jvm_old_gc_time

Max. JVM Old GC Duration

Maximum accumulated JVM Old GC duration of nodes in a CSS cluster.

Unit: ms

≥ 0 ms

CSS cluster

1 minute

max_jvm_old_gc_count

Max. JVM Old GC Count

Maximum accumulated JVM Old GC count of nodes in a CSS cluster.

≥ 0

CSS cluster

1 minute

total_fs_size

Total Size of File Systems

Total size of file systems in a CSS cluster.

Unit: byte

≥ 0 bytes

CSS cluster

1 minute

free_fs_size

Available Size of File Systems

Available size of file systems in a CSS cluster.

Unit: byte

≥ 0 bytes

CSS cluster

1 minute

max_cpu_usage

Max. CPU Usage

Maximum node CPU usage in a CSS cluster.

Unit: %

0-100%

CSS cluster

1 minute

max_cpu_time_of_jvm_process

Max. CPU Time of JVM Process

Maximum accumulated CPU usage duration of node JVM processes in a CSS cluster.

Unit: ms

≥ 0 ms

CSS cluster

1 minute

max_virtual_memory_size_of_jvm_process

Max. Virtual Memory Size of JVM Process

Maximum virtual memory size of node JVM processes in a CSS cluster.

Unit: byte

≥ 0 bytes

CSS cluster

1 minute

max_current_opened_http_count

Current Max. Opened HTTP Connections

Maximum number of HTTP connections that are currently open for nodes in a CSS cluster.

≥ 0

CSS cluster

1 minute

max_total_opened_http_count

Total Max. Opened HTTP Connections

Maximum number of HTTP connections that were open for nodes in a CSS cluster.

≥ 0

CSS cluster

1 minute

indices_count

Indexes

Number of indexes in a CSS cluster

≥ 0

CSS cluster

1 minute

total_shards_count

Shards

Number of shards in a CSS cluster

≥ 0

CSS cluster

1 minute

primary_shards_count

Primary Shards

Number of primary shards in a CSS cluster

≥ 0

CSS cluster

1 minute

docs_count

Documents

Number of documents in a CSS cluster

≥ 0

CSS cluster

1 minute

docs_deleted_count

Deleted Documents

Number of documents deleted in a CSS cluster

≥ 0

CSS cluster

1 minute

nodes_count

Nodes

Number of nodes in a CSS cluster

≥ 0

CSS cluster

1 minute

data_nodes_count

Data Nodes

Number of data nodes in a CSS cluster

≥ 0

CSS cluster

1 minute

coordinating_nodes_count

Coordinating Nodes

Number of coordinating nodes in a CSS cluster

≥ 0

CSS cluster

1 minute

master_nodes_count

Master Nodes

Number of master nodes in a CSS cluster

≥ 0

CSS cluster

1 minute

ingest_nodes_count

Client Nodes

Number of client nodes in a CSS cluster

≥ 0

CSS cluster

1 minute

max_load_average

Max. Node Load

Maximum number of average queuing tasks per minute on nodes in a cluster.

≥ 0

CSS cluster

1 minute

avg_cpu_usage

Avg. CPU Usage

Average node CPU usage in a CSS cluster.

Unit: %

0-100%

CSS cluster

1 minute

avg_load_average

Avg. Node Load

Average number of queuing tasks per minute on nodes in a CSS cluster.

≥ 0

CSS cluster

1 minute

avg_jvm_heap_usage

Avg. JVM Heap Usage

Average node JVM heap usage in a CSS cluster.

Unit: %

0-100%

CSS cluster

1 minute

max_open_file_descriptors

Max. Open File Descriptors

Maximum number of node file descriptors that are currently open in a CSS cluster.

≥ 0

CSS cluster

1 minute

avg_open_file_descriptors

Avg. Open File Descriptors

Average number of node file descriptors that are currently open in a CSS cluster.

≥ 0

CSS cluster

1 minute

sum_max_file_descriptors

Max. Allowed File Descriptors

Maximum number of allowed node file descriptors in a CSS cluster.

≥ 0

CSS cluster

1 minute

sum_open_file_descriptors

Open File Descriptors

Number of node file descriptors that are currently open in a cluster.

≥ 0

CSS cluster

1 minute

sum_thread_pool_write_queue

Tasks in Write Queue

Number of job queues in a write thread pool

≥ 0

CSS cluster

1 minute

sum_thread_pool_search_queue

Tasks in Search Queue

Total number of queuing tasks in the search thread pools of nodes in the CSS cluster.

≥ 0

CSS cluster

1 minute

sum_thread_pool_force_merge_queue

Tasks in ForceMerge Queue

Total number of queuing tasks in the force merge thread pools of nodes in the CSS cluster.

≥ 0

CSS cluster

1 minute

sum_thread_pool_write_rejected

Rejected Tasks in Write Queue

Total number of rejected tasks in the write thread pools of nodes in the CSS cluster.

≥ 0

CSS cluster

1 minute

sum_thread_pool_search_rejected

Rejected Tasks in Search Queue

Total number of rejected tasks in the search thread pools of nodes in the CSS cluster.

≥ 0

CSS cluster

1 minute

sum_thread_pool_force_merge_rejected

Rejected Tasks in ForceMerge Queue

Total number of rejected tasks in the force merge thread pools of nodes in the CSS cluster.

≥ 0

CSS cluster

1 minute

max_thread_pool_search_queue

Max. Tasks in Search Queue

Maximum number of queuing tasks in the search thread pools of nodes in a CSS cluster.

≥ 0

CSS cluster

1 minute

max_thread_pool_force_merge_queue

Max. Tasks in ForceMerge Queue

Maximum number of queuing tasks in the force merge thread pools of nodes in a CSS cluster.

≥ 0

CSS cluster

1 minute

sum_thread_pool_write_threads

Size of Write Thread Pool

Total size of the write thread pools of nodes in the CSS cluster.

≥ 0

CSS cluster

1 minute

sum_thread_pool_search_threads

Size of Search Thread Pool

Total size of the search thread pools of nodes in the CSS cluster.

≥ 0

CSS cluster

1 minute

sum_thread_pool_force_merge_threads

Size of ForceMerge Thread Pool

Total size of the force merge thread pools of nodes in the CSS cluster.

≥ 0

CSS cluster

1 minute

avg_thread_pool_write_queue

Avg. Tasks in Write Queue

Average number of queuing tasks in the write thread pools of nodes in a CSS cluster.

≥ 0

CSS cluster

1 minute

avg_thread_pool_search_queue

Avg. Tasks in Search Queue

Average number of queuing tasks in the search thread pools of nodes in a CSS cluster.

≥ 0

CSS cluster

1 minute

avg_thread_pool_force_merge_queue

Avg. Tasks in ForceMerge Queue

Average number of queuing tasks in the force merge thread pools of nodes in the CSS cluster.

≥ 0

CSS cluster

1 minute

avg_thread_pool_search_threads

Avg. Size of Search Thread Pool

Average size of the search thread pool of a node in a CSS cluster.

≥ 0

CSS cluster

1 minute

avg_thread_pool_write_threads

Avg. Size of Write Thread Pool

Average size of the write thread pool of a node in a CSS cluster.

≥ 0

CSS cluster

1 minute

avg_thread_pool_force_merge_threads

Avg. Size of ForceMerge Thread Pool

Average size of the force merge thread pool of a node in a CSS cluster.

≥ 0

CSS cluster

1 minute

avg_thread_pool_write_rejected

Avg. Rejected Tasks in Write Queue

Average number of rejected tasks in the write thread pool of a node in a CSS cluster.

≥ 0

CSS cluster

1 minute

min_free_fs_size

Min. Available Storage Space

Minimum available storage space of nodes in a CSS cluster.

Unit: byte

≥ 0 bytes

CSS cluster

1 minute

avg_jvm_old_gc_count

Avg. GCs of Old-Generation JVM

Average number of old-generation garbage collections of nodes in a CSS cluster.

≥ 0

CSS cluster

1 minute

avg_jvm_old_gc_time

Avg. GC Duration of Old-Generation JVM

Average old-generation garbage collection duration of nodes in a CSS cluster.

Unit: ms

≥ 0 ms

CSS cluster

1 minute

avg_jvm_young_gc_count

Avg. GCs of Young-Generation JVM

Average number of young-generation garbage collections of nodes in a CSS cluster.

≥ 0

CSS cluster

1 minute

avg_jvm_young_gc_time

Avg. GC Duration of Young-Generation JVM

Average young-generation garbage collection duration of nodes in a CSS cluster.

Unit: ms

≥ 0 ms

CSS cluster

1 minute

avg_max_file_descriptors

Avg. Maximum Allowed File Descriptors

Average value of the maximum number of allowed file descriptors on each node in a CSS cluster.

≥ 0

CSS cluster

1 minute

avg_mem_free_in_bytes

Avg. Available Memory

Average unused memory capacity of nodes in a CSS cluster.

Unit: byte

≥ 0 bytes

CSS cluster

1 minute

avg_mem_free_percent

Avg. Available Memory Percentage

Average percentage of unused memory of nodes in a CSS cluster.

Unit: %

0-100%

CSS cluster

1 minute

avg_mem_used_in_bytes

Avg. Used Memory

Average used memory of nodes in a CSS cluster.

Unit: byte

≥ 0 bytes

CSS cluster

1 minute

avg_mem_used_percent

Avg. Used Memory Percentage

Average percentage of used memory of nodes in a CSS cluster.

Unit: %

0-100%

CSS cluster

1 minute

max_mem_free_in_bytes

Max. Available Memory

Maximum unused memory of nodes in a CSS cluster.

Unit: byte

≥ 0 bytes

CSS cluster

1 minute

max_mem_free_percent

Max. Available Memory Percentage

Maximum percentage of unused memory of nodes in a CSS cluster.

Unit: %

0-100%

CSS cluster

1 minute

max_mem_used_in_bytes

Max. Used Memory

Maximum used memory of nodes in a CSS cluster.

Unit: byte

≥ 0 bytes

CSS cluster

1 minute

max_mem_used_percent

Max. Used Memory Percentage

Maximum percentage of used memory of nodes in a CSS cluster.

Unit: %

0-100%

CSS cluster

1 minute

sum_jvm_old_gc_count

Total GCs of Old-Generation JVM

Number of old-generation garbage collections of nodes in a CSS cluster.

≥ 0

CSS cluster

1 minute

sum_jvm_old_gc_time

Total GC Duration of Old-Generation JVM

Total old-generation garbage collection duration of nodes in the CSS cluster.

Unit: ms

≥ 0ms

CSS cluster

1 minute

sum_jvm_young_gc_count

Total GCs of Young-Generation JVM

Number of young-generation garbage collections of nodes in a CSS cluster.

≥ 0

CSS cluster

1 minute

sum_jvm_young_gc_time

Total GC Duration of Young-Generation JVM

Total young-generation garbage collection duration of nodes in the CSS cluster.

Unit: ms

≥ 0 ms

CSS cluster

1 minute

sum_current_opened_http_count

Currently Open HTTP Connections

Number of HTTP connections that are open on nodes in a CSS cluster.

≥ 0

CSS cluster

1 minute

sum_total_opened_http_count

Historical Open HTTP Connections

Number of HTTP connections that were open on nodes in a CSS cluster.

≥ 0

CSS cluster

1 minute

IndexingLatency

Average Index Latency

Average time required for a shard to complete an index operation.

Unit: ms

≥ 0 ms

CSS cluster

1 minute

IndexingRate

Average Index Rate

Average number of index operations per second in a cluster.

≥ 0

CSS cluster

1 minute

SearchLatency

Average Search Latency

Average time required for a segment to complete the search operation.

Unit: ms

≥ 0 ms

CSS cluster

1 minute

SearchRate

Average QPS

Average queries per second (QPS) in a cluster.

≥ 0

CSS cluster

1 minute

Dimension

Table 2 Dimension description

Key

Value

cluster_id

CSS cluster