Updated on 2023-08-02 GMT+08:00

KafkaConsumer Monitoring

This section describes the types, names, and meanings of KafkaConsumer metrics collected by APM.

Table 1 KafkaConsumer monitoring collection parameters

Parameter

Data Type

Application Type

Default

Supported Start Agent Version

Supported End Agent Version

Description

Kafka Slow Request Threshold

integer

JAVA

800

2.1.14

-

The sampling ratio will increase if the slow request threshold is crossed.

Kafka Consumption Method Configuration

obj_array

JAVA

-

2.1.14

-

Kafka consumption method configuration

Table 2 KafkaConsumer metrics

Category

Metric

Name

Description

Unit

Data Type

Default Aggregation Mode

Topic (topic: Kafka topic monitoring data)

id

id

Client ID and IP address

-

ENUM

LAST

topic

topic

Kafka topic name

-

ENUM

LAST

bytesConsumedRate

Bytes Consumed/s

Number of bytes consumed per second

Byte

INT

AVG

fetchSizeAvg

Avg. Bytes Fetched

Average number of bytes fetched for a request

Byte

INT

AVG

fetchSizeMax

Max. Bytes Fetched

Maximum number of bytes fetched for a request

Byte

INT

MAX

recordsConsumedRate

Messages Consumed/s

Number of messages consumed per second

-

INT

AVG

recordsPerRequestAvg

Avg. Messages of Single Request

Average number of messages of a single request

-

INT

AVG

seqIds

Producer-generated SN

Sequence number generated by the producer

-

STRING

LAST

recordConsumedTotal

Total Consumption Times

Total number of consumption times

-

INT

SUM

bytesConsumedTotal

Total Consumed Bytes

Total number of bytes that have been consumed

-

INT

SUM

Fetch monitoring (fetch: Kafka fetch monitoring data)

id

id

Client ID and IP address

-

ENUM

LAST

bytesConsumedRate

Bytes Consumed/s

Number of bytes consumed per second

Byte

INT

AVG

fetchLatencyAvg

Avg. Request Latency

Average request latency

ms

INT

AVG

fetchLatencyMax

Max. Request Latency

Maximum request latency

ms

INT

MAX

fetchRate

Requests/s

Number of requests per second

-

INT

AVG

fetchSizeAvg

Avg. Bytes Fetched

Average number of bytes fetched for a request

Byte

INT

AVG

fetchSizeMax

Max. Bytes Fetched

Maximum number of bytes fetched for a request

Byte

INT

MAX

recordsConsumedRate

Messages Consumed/s

Number of messages consumed per second

-

INT

AVG

recordsLagMax

Max. Accumulated Messages

Maximum number of accumulated messages

-

INT

MAX

recordsPerRequestAvg

Avg. Messages of Single Request

Average number of messages of a single request

-

INT

AVG

seqIds

Producer-generated SN

Sequence number generated by the producer

-

STRING

LAST

recordConsumedTotal

Total Consumption Times

Total number of consumption times

-

INT

SUM

bytesConsumedTotal

Total Consumed Bytes

Total number of bytes that have been consumed

-

INT

SUM

Partition (partition: Kafka partition data)

id

id

Client ID and IP address

-

ENUM

LAST

partition

partition

Kafka partition name

-

ENUM

LAST

recordsLag

Accumulated Messages

Number of accumulated messages

-

INT

LAST

recordsLagAvg

Avg. Accumulated Messages

Average number of accumulated messages

-

INT

AVG

recordsLagMax

Max. Accumulated Messages

Maximum number of accumulated messages

-

INT

MAX

seqIds

Producer-generated SN

Sequence number generated by the producer

-

STRING

LAST

Kafka consumption method monitoring (consumer)

method

Method

Consumption method

-

ENUM

LAST

concurrentMax

Max. Concurrency

Maximum concurrency

-

INT

MAX

errorCount

Errors

Number of errors

-

INT

SUM

invokeCount

Calls

Number of calls

-

INT

SUM

lastError

Error Message

Error details

-

STRING

LAST

maxTime

Max. RT

Maximum response time in a collection period

-

INT

MAX

range1

0–10 ms

Number of requests with 0–10 ms response time

-

INT

SUM

range2

10–100 ms

Number of requests with 10–100 ms response time

-

INT

SUM

range3

100–500 ms

Number of requests with 100–500 ms response time

-

INT

SUM

range4

500–1000 ms

Number of requests with 500–1000 ms response time

-

INT

SUM

range5

1–10s

Number of requests with 1–10s response time

-

INT

SUM

range6

> 10s

Number of requests with more than 10s response time

-

INT

SUM

totalTime

Total RT

Total response time

-

INT

SUM

KafkaConsumer summary (total)

recordConsumedTotal

Total Consumption Times

Total number of consumption times

-

INT

SUM

bytesConsumedTotal

Total Consumed Bytes

Total number of bytes that have been consumed

-

INT

SUM

recordsLag

Total Accumulated Messages

Total number of messages that have been accumulated

-

INT

LAST

Exception (exception: exception statistics about Kafka consumption)

causeType

Exception Class

Exception class

-

ENUM

LAST

exceptionType

Exception Class

Exception class

-

ENUM

LAST

count

Count

Number of exceptions

-

INT

SUM

message

Exception Message

Exception message

-

STRING

LAST

stackTrace

Exception Stack

Exception stack

-

CLOB

LAST