Monitoring Metrics for Logstash Clusters in Cloud Eye

You can use Cloud Eye to monitor CSS cluster metrics in real time and quickly handle exceptions.

Function

This section describes CSS metrics that can be monitored by Cloud Eye as well as their namespaces and dimensions. You can query metrics and alerts generated by CSS on the Cloud Eye console or using an API.

If the configuration center of a Logstash cluster does not have any records of operations on the pipeline list, the monitoring records of this Logstash cluster are empty.
When the Events data of a pipeline changes dynamically, the monitoring data changes accordingly. When a pipeline task is being started or stopped, or the Events data is stable, the monitoring data remains unchanged.

Cloud Eye can monitor dimensions (or objects) nested to a maximum depth of four levels (levels 0 to 3). 3 is the deepest level. For example, if the monitored dimension of a metric is cluster_id,instance_id, cluster_id indicates level 0 and instance_id indicates level 1.

Namespaces

SYS.ES

CSS.CUSTOM

Cluster Monitoring Metrics

Accumulated value: The value is accumulated from when a node is started. After the node is restarted, the value is reset to zero and starts accumulating again.

**Table 1** Logstash cluster monitoring metrics
Metric ID	Metric	Description	Value Range	Unit	Number System	Dimension	Monitoring Interval (Raw)
max_jvm_heap_usage	Max. JVM Heap Usage	Maximum JVM heap usage of nodes in a CSS cluster	0~100	%	N/A	cluster_id	1 minute
max_jvm_young_gc_time	Max. JVM Young GC Duration	Maximum accumulated JVM Young GC duration of nodes in a CSS cluster	≥ 0	ms	N/A	cluster_id	1 minute
max_jvm_young_gc_count	Max. JVM Young GC Count	Maximum accumulated JVM Young GC count of nodes in a CSS cluster	≥ 0	Count	N/A	cluster_id	1 minute
max_jvm_old_gc_time	Max. JVM Old GC Duration	Maximum accumulated JVM Old GC duration of nodes in a CSS cluster	≥ 0	ms	N/A	cluster_id	1 minute
max_jvm_old_gc_count	Max. JVM Old GC Count	Maximum accumulated JVM Old GC count of nodes in a CSS cluster.	≥ 0	Count	N/A	cluster_id	1 minute
max_cpu_usage	Max. CPU Usage	Maximum node CPU usage in a CSS cluster	0~100	%	N/A	cluster_id	1 minute
max_load_average	Max. Node Load	Maximum number of average queued tasks per minute on nodes in a cluster	≥ 0	Count	N/A	cluster_id	1 minute
avg_cpu_usage	Avg. CPU Usage	Average node CPU usage in a CSS cluster.	0~100	%	N/A	cluster_id	1 minute
avg_load_average	Avg. Node Load	Average number of queued tasks per minute on nodes in a CSS cluster	≥ 0	Count	N/A	cluster_id	1 minute
avg_jvm_heap_usage	Avg. JVM Heap Usage	Average node JVM heap usage in a CSS cluster	0~100	%	N/A	cluster_id	1 minute
avg_jvm_old_gc_count	Avg. GCs of Old-Generation JVM	Average number of old-generation garbage collections of nodes in a CSS cluster	≥ 0	Count	N/A	cluster_id	1 minute
avg_jvm_old_gc_time	Avg. GC Duration of Old-Generation JVM	Average old-generation garbage collection duration of nodes in a CSS cluster	≥ 0	ms	N/A	cluster_id	1 minute
avg_jvm_young_gc_count	Avg. GCs of Young-Generation JVM	Average number of young-generation garbage collections of nodes in a CSS cluster	≥ 0	Count	N/A	cluster_id	1 minute
avg_jvm_young_gc_time	Avg. GC Duration of Young-Generation JVM	Average young-generation garbage collection duration of nodes in a CSS cluster	≥ 0	ms	N/A	cluster_id	1 minute
sum_events_in	Total Records Passed Through the Input Plug-in	Total number of records that have passed through the input plugin on all the nodes in a cluster	≥ 0	Count	N/A	cluster_id	1 minute
sum_events_filtered	Total Records Passed Through the Filter Plug-in	Total number of records that have passed through the filter plugin on all the nodes in a cluster	≥ 0	Count	N/A	cluster_id	1 minute
sum_events_out	Total Records Passed Through the Out Plug-in	Total number of records that have passed through the out plugin on all the nodes in a cluster	≥ 0	Count	N/A	cluster_id	1 minute

Node Monitoring Metrics

**Table 2** Logstash node monitoring metrics
Metric ID	Metric	Description	Value Range	Unit	Number System	Dimension	Monitoring Interval (Raw)
jvm_heap_usage	JVM Heap Usage	JVM heap memory usage of a node.	0~100	%	N/A	cluster_id,instance_id	1 minute
cpu_usage	CPU Usage	CPU usage.	0~100	%	N/A	cluster_id,instance_id	1 minute
load_average	Average Load	Average number of queued tasks per minute on a node	≥ 0	Count	N/A	cluster_id,instance_id	1 minute
jvm_old_gc_count	Total GCs of Old-Generation JVM	Number of old-generation garbage collection times	≥ 0	Count	N/A	cluster_id,instance_id	1 minute
jvm_old_gc_time	Total GC Duration of Old-Generation JVM	Time spent on old-generation garbage collection	≥ 0	ms	N/A	cluster_id,instance_id	1 minute
jvm_young_gc_count	Total GCs of Young-Generation JVM	Number of young-generation garbage collection times	≥ 0	Count	N/A	cluster_id,instance_id	1 minute
jvm_young_gc_time	GC Duration of Young-Generation JVM	Time spent on young-generation garbage collection	≥ 0	ms	N/A	cluster_id,instance_id	1 minute
events_in	Records Passed Through the Input Plug-in on Node	Number of data records that have passed through the input plugin on the current node	≥ 0	Count	N/A	cluster_id,instance_id	1 minute
events_filtered	Records Passed Through the Filter Plug-in on Node	Number of records that have passed through the filter plugin on the current node	≥ 0	Count	N/A	cluster_id,instance_id	1 minute
events_out	Records Passed Through the Out Plug-in on Node	Number of records that have passed through the out plugin on the current node	≥ 0	Count	N/A	cluster_id,instance_id	1 minute

Logstash Pipeline Monitoring Metrics

**Table 3** Logstash pipeline monitoring metrics
Metric ID	Metric	Description	Value Range	Unit	Number System	Dimension	Monitoring Interval (Raw)
logstash_pipeline_events_in	Records Passed Through the Input Plug-in into Pipeline	Number of records that have passed through the input plug-in during the current pipeline monitoring period	≥ 0	Count	N/A	cluster_id,instance_id,pipeline_name or cluster_id,pipeline_name	1 minute
logstash_pipeline_events_filtered	Records Passed Through the Filter Plug-in	Number of records that have passed through the filtered plug-in during the current pipeline monitoring period	≥ 0	Count	N/A	cluster_id,instance_id,pipeline_name or cluster_id,pipeline_name	1 minute
logstash_pipeline_events_out	Records Passed Through the Out Plug-in Out of Pipeline	Number of records that have passed through the out plug-in during the current pipeline monitoring period	≥ 0	Count	N/A	cluster_id,instance_id,pipeline_name or cluster_id,pipeline_name	1 minute

If an object is in a hierarchical system, specify the monitored dimension in hierarchical form when you use an API to query the metrics of this object.

For example, to query the CPU usage (cpu_usage) of a Logstash cluster node in CSS, the dimension information of this metric is cluster_id,instance_id, where cluster_id indicates level 0 and instance_id level 1.

To query a single metric by calling an API, the instance_id dimension is used as follows:
```
dim.0=cluster_id,3d65c1ac-9a9f-4c5f-a054-35184a087bb2&dim.1=instance_id,6666cd76f96956469e7be39d750cc7d9
```
where, 3d65c1ac-9a9f-4c5f-a054-35184a087bb2 and 6666cd76f96956469e7be39d750cc7d9 are the values of cluster_id and instance_id, respectively. For how to obtain these values, see Dimensions.
To query multiple metrics by calling an API, the instance_id dimension is used as follows:
```
"dimensions": [
	{
		"name": "cluster_id",
		"value": "3d65c1ac-9a9f-4c5f-a054-35184a087bb2"
	},
	{
		"name": "instance_id",
		"value": "6666cd76f96956469e7be39d750cc7d9"
	}
]
```
where, 3d65c1ac-9a9f-4c5f-a054-35184a087bb2 and 6666cd76f96956469e7be39d750cc7d9 are the values of cluster_id and instance_id, respectively. For how to obtain the values, see Dimensions.

Dimensions

**Table 4** Dimension description
Key	Value
cluster_id	Cluster ID. You can obtain this value from the clusters[].id field in the response body of the Querying the Cluster List API.
instance_id	Cluster node ID. You can obtain this value from the clusters[].instances[].id field in the response body of the Querying the Cluster List API.
pipeline_name	Logstash pipeline name. You can obtain this value from the pipelines[].name field in the response body of the Querying the Pipeline List.