ClickHouse Cluster Supported Metrics

Description

Monitoring is critical to ensure CloudTable reliability, availability, and performance. You can monitor the running status of CloudTable servers. This section describes the metrics that can be monitored by CES as well as their namespaces and dimensions.

Namespace

SYS.CloudTable

Monitoring metrics

**Table 1** ClickHouse monitoring metric description
Metric	Name	Description	Minimum Unit	Monitoring Period
cpu_usage	CPU Utilization Rate	CPU utilization rate of each node.	%	60
memory_usage	Memory Usage	Memory usage of each node.	Byte	60
memory_usage_ratio	Memory Utilization Rate	Memory utilization rate of each node.	%	60
disk_usage	Disk Usage	Disk space used by each node.	Byte	60
disk_usage_ratio	Disk Utilization Rate	Disk utilization rate of each node.	%	60
network_throughput_inbound_rate	Inbound Throughput	Inbound data volume over network of each node per second.	Byte/s	60
network_throughput_outgoing_rate	Outbound Throughput	Outbound data volume over network of each node per second.	Byte/s	60
clickhouse_process_survival	ClickHouse Process Survival Status	Heartbeat check of the ClickHouse process.	-	60
number_of_mutations	Mutations	Number of running modification tasks on each node.	Count	60
number_of_run_merges	Running Merges	Number of running merge tasks.	Count	60
number_of_merges_thread	Merge Threads	Number of threads being merged.	Count	60
number_of_zookeeper_requests	ZooKeeper Requests	Number of ZooKeeper requests on the node at the current time.	Count	60
number_of_zk_session	ZooKeeper Sessions	Number of connected ZooKeeper sessions on the node at the current time.	Count	60
number_of_zookeeper_watches	ZooKeeper Watches	Number of connected ZooKeeper watches on the node at the current time.	Count	60
number_of_zookeeper_nodes	ZooKeeper Nodes	Number of temporary nodes created in ZooKeeper.	Count	60
number_of_parts	Parts	Number of parts.	Count	60
clickhouse_zookeeper_disconnection	ZooKeeper Connection Status	ZooKeeper connection status.	-	60
number_of_threads_holding_read_locks	Threads Holding the Read Lock	Number of threads that hold read locks.	Count	60
number_of_threads_waiting_to_be_read	Threads Waiting to Be Read	Number of threads waiting to be read.	Count	60
number_of_threads_waiting_to_be_written	Number of Threads Waiting to Be Written	Number of threads waiting to be written.	Count	60
number_of_threads_holding_write_locks	Threads Holding the Write Lock	Number of threads that hold write locks.	Count	60
number_of_active_tasks_background_processing_pool	Active Tasks in Background Processing Pool	Number of active tasks in the background processing pool.	Count	60
maximum_number_of_active_blocks	Max. Active Blocks	Maximum number of active blocks of the ClickHouse instance.	Count	60
number_of_consistency_checkdata_blocks	Data Blocks for Consistency Check	Number of data blocks for consistency check on a ClickHouse instance.	Count	60
extracting_the_number_of_data_chunks_from_the_replica	Data Blocks Being Extracted from a Replica	Number of data blocks that are being extracted from a replica by a ClickHouse instance.	Count	60
number_of_data_chunks_being_sent_to_the_replica	Data Blocks Being Sent to a Replica	Number of data blocks on a ClickHouse instance that are being sent to a replica.	Count	60
number_of_readonly_replicas	Read-Only Replicas	Number of read-only replicas of a ClickHouse instance.	Count	60
number_of_inserted_into_all_tables	Inserted Rows	Number of rows inserted into all tables in a ClickHouse instance.	Count	60
number_of_open_read_files	Opened Read Files	Number of opened read files on a ClickHouse instance.	Count	60
number_of_open_write_files	Opened Write Files	Number of opened write files on a ClickHouse instance.	Count	60
total_file_opens	Opened Files	Number of opened files on a ClickHouse instance.	Count	60
number_of_read_file_descriptors	Read File Descriptors	Number of file descriptors that have been read.	Count	60
number_of_written_file_descriptors	Write File Descriptors	Number of file descriptors that have been written.	Count	60
instance_process_running_duration	Instance Process Running Duration	Running duration of the process on a ClickHouse instance.	s	60
number_of_threads_running_in_global_thread_pool	Threads of Running Tasks in Global Thread Pool	Number of threads of running tasks in the global thread pool of a ClickHouse instance.	Count	60
number_of_threads_in_global_thread_pool	Threads in Global Thread Pool	Number of threads in the global thread pool of a ClickHouse instance.	Count	60
number_of_threads_in_local_thread_pool	Threads in Local Thread Pool	Number of threads in the local thread pool of a ClickHouse instance.	Count	60
number_of_threads_waiting_locked	Threads to Lock in Context	Number of threads waiting to be locked in context on a ClickHouse instance.	Count	60
number_of_threads_running_in_local_thread_pool	Threads of Running Tasks in Local Thread Pool	Number of threads of running tasks in the local thread pool of a ClickHouse instance.	Count	60
number_of_tcp_connections	TCP Connections	Number of TCP connections.	Count	60
number_of_http_connections	HTTP Connections	Number of HTTP connections.	Count	60
number_of_databases	Databases	Number of databases on each node.	Count	60
number_of_tables	Tables	Number of local tables on each node.	Count	60
write_size_per_second	Write Size per Second	Amount of data written to each node per unit time.	Byte	60
number_of_running_queries	Running Queries	Number of running queries on each node.	Count	60
number_of_query_threads	Query Threads	Number of running queries on each node.	Count	60
number_of_distributed_ddls	Distributed DDLs	Number of distributed DDL statements.	Count	60
number_of_distributed_table_files	Distributed Tables	Number of tables to which data is to be inserted.	Count	60
number_of_concurrency	Concurrency of Each ClickHouse Node	Concurrency of each ClickHouse node.	Count	60
zookeeper_quota_percentage	ZooKeeper Quantity Percentage	Quota of the ZooKeeper nodes used by ClickHouse.	%	60
zookeeper_process_is_alive	ZooKeeper Process Survival Status	ZooKeeper heartbeat check. 1 indicates that the ZooKeeper heartbeat is normal, and 0 indicates that it is abnormal.	-	60
number_of_global_sessions	Global Sessions	Number of global sessions.	Count	60
number_of_rejected_connections	Refused Connections	Number of connections refused by Zookeeper.	Count	60
number_of_request_submission_queues	Requests to Submit in a Queue	Number of requests submitted in a queue.	Count	60
waiting_time_of_the_preprocessing_queue	Wait Time of Request in Preprocessing Queue	Time that a request in the preprocessing queue waits to be processed.	s	60
number_of_zk_watches	ZooKeeper Watches	Number of ZooKeeper watches at the current time.	Count	60
zk_heapmemory_usage	Heap Memory Used	Heap memory size used by ZooKeeper process.	Byte	60
zk_directmemory_usage	Used Direct Memory	Direct memory size used by ZooKeeper process.	Byte	60
zk_heapmemory_usage_ratio	Heap Memory Usage	Heap memory usage of the ZooKeeper service.	%	60
zk_directmemory_usage_ratio	Direct Memory Usage	Direct memory usage of the ZooKeeper service.	%	60
cluster_number_of_sent_packets	ZooKeeper Sent Packets	Number of packets sent by a ZooKeeper node.	Count	60
cluster_number_of_received_packets	ZooKeeper Received Packets	Number of packets received by a ZooKeeper node.	Count	60
cluster_preprocessing_time	Preprocessing Duration	Preprocessing time.	s	60
rows_written_per_second	Rows Written per Second	Number of rows written to each node per unit time.	Count	60
number_of_failed_queries	Failed Queries	Number of failed query statements.	Count	60
number_of_failed_insert_queries	Failed Insert Queries	Number of failed insert statements.	Count	60
number_of_failed_select_query_messages	Failed Select Queries	Number of failed select statements.	Count	60
number_of_delayed_inserts	Delayed Inserts	Number of delayed insertions on each node.	Count	60
number_of_rows	Rows	Number of mergetree engine fields on each node.	Count	60
disk_throughput_read_rate	Disk Read Throughput	Disk read throughput.	Byte	60
disk_throughput_write_rate	Disk Write Throughput	Disk write throughput.	Byte	60
cmdForUsedStorageRate	Used Storage Rate	Ratio of the used storage space to the total storage space in the cluster	%	60