Prometheus Server View

Prometheus with local data storage enabled scrapes the metrics of all hosts and components. The metrics are then reported to and stored on AOM or a third-party monitoring platform. The Prometheus Server view displays some built-in metrics provided by Prometheus, which can be used to monitor and measure system performance and status.

Metric Description

You can view Prometheus Server metrics in the following table.

Figure 1 Prometheus Server metrics
Click to enlarge

**Table 1** Prometheus Server metrics
Metric	Unit	Description
Target Sync	second	Collection latency of the target.
Targets	N/A	Total number of metrics collected by the target.
Average Scrape Interval	second	Interval for collecting metrics.
Scrape Failures	N/A	The number of collection failures.
Appended Samples Rate	N/A	The rate at which samples are added to the head.
Head Series	N/A	The number of series in the head.
Head Chunks	N/A	The number of head blocks.
Query Rate	N/A	The number of Prometheus queries per second.
P90 Query Duration	second	Duration for querying 90% of operations on different shards.
Remote Sample Lag Ratio	second	Percentage of the highest timestamp of samples stored in WAL.
Remote Write Traffic	byte/s	Remote write rate.
Current Shards	N/A	The number of shards used for parallel sending to the remote storage.
Max Shards	N/A	Maximum number of shards that the queue is allowed to run.
Min Shards	N/A	Minimum number of shards that the queue is allowed to run.
Desired Shards	N/A	The number of shards that the queue wants to run based on the percentage of input samples to output samples.
Shard Capacity	N/A	Capacity of each shard of the queue used for parallel sending to the remote storage.
Pending Samples	N/A	Capacity of each shard of the queue used for parallel sending to the remote storage.
Current TSDB Segment	N/A	WAL segment index that TSDB is currently writing to.
Current Segment of Remote Write	N/A	Current segment from which the WAL watcher is reading records.
Sample Discard Rate	N/A	The rate at which samples are discarded after being read from WAL before being sent via remote write.
Sample Failure Rate	N/A	Rate of samples that failed to be sent to remote storage due to unrecoverable errors.
Sample Retry Rate	N/A	Rate of samples that failed to be sent to remote storage due to recoverable errors and were resent.
Retry Rate of Enqueuing	N/A	Retry rate of enqueuing failed due to full shard queue.

Metric List

The following is the metric list of the Prometheus Server view.

**Table 2** Metric description
Metric	Type	Description
prometheus_target_sync_length_seconds_sum	Summary	Collection latencies of different targets.
prometheus_sd_discovered_targets	Gauge	The number of metrics collected by different targets.
prometheus_target_interval_length_seconds_sum	Summary	Collection interval.
prometheus_target_scrapes_exceeded_body_size_limit_total	Counter	The number of collection failures.
prometheus_tsdb_head_samples_appended_total	Counter	Total number of samples added to the head.
prometheus_tsdb_head_series	Gauge	The number of series in the head block.
prometheus_tsdb_head_chunks	Gauge	The number of chunks in the head block.
prometheus_engine_query_duration_seconds_count	Counter	The number of queries.
prometheus_engine_query_duration_seconds	Counter	Time taken by queries to complete, in seconds.
prometheus_remote_storage_highest_timestamp_in_seconds	Gauge	Latest timestamp in the remote storage.
prometheus_remote_storage_queue_highest_sent_timestamp_seconds	Gauge	Latest timestamp in the Prometheus shard.
prometheus_remote_storage_bytes_total	Counter	Total number of bytes of data (non-metadata) sent by the queue after compression.
prometheus_remote_storage_shards	Gauge	The number of shards used for parallel sending to the remote storage.
prometheus_remote_storage_shards_max	Gauge	Maximum number of shards that the queue is allowed to run.
prometheus_remote_storage_shards_min	Gauge	Minimum number of shards that the queue is allowed to run.
prometheus_remote_storage_shards_desired	Gauge	The number of shards that the queue wants to run based on the percentage of input samples to output samples.
prometheus_remote_storage_shard_capacity	Gauge	Capacity of each shard of the queue used for parallel sending to the remote storage.
prometheus_remote_storage_pending_samples	Gauge	The number of pending samples in queue shards to be sent to remote storage.
prometheus_tsdb_wal_segment_current	Gauge	WAL segment index that TSDB is currently writing to.
prometheus_wal_watcher_current_segment	Gauge	Current segment from which the WAL is reading records.
prometheus_remote_storage_dropped_samples_total	Gauge	The rate at which samples are discarded after being read from WAL before being sent via remote write.
prometheus_remote_storage_failed_samples_total	Gauge	Rate of samples that failed to be sent to remote storage due to unrecoverable errors.
prometheus_remote_storage_retried_samples_total	Gauge	The number of samples that failed to be sent to remote storage due to recoverable errors and were resent.
prometheus_remote_storage_enqueue_retries_total	Gauge	The number of retries upon enqueuing failed due to full shard queue.