Compute
Elastic Cloud Server
Huawei Cloud Flexus
Bare Metal Server
Auto Scaling
Image Management Service
Dedicated Host
FunctionGraph
Cloud Phone Host
Huawei Cloud EulerOS
Networking
Virtual Private Cloud
Elastic IP
Elastic Load Balance
NAT Gateway
Direct Connect
Virtual Private Network
VPC Endpoint
Cloud Connect
Enterprise Router
Enterprise Switch
Global Accelerator
Management & Governance
Cloud Eye
Identity and Access Management
Cloud Trace Service
Resource Formation Service
Tag Management Service
Log Tank Service
Config
OneAccess
Resource Access Manager
Simple Message Notification
Application Performance Management
Application Operations Management
Organizations
Optimization Advisor
IAM Identity Center
Cloud Operations Center
Resource Governance Center
Migration
Server Migration Service
Object Storage Migration Service
Cloud Data Migration
Migration Center
Cloud Ecosystem
KooGallery
Partner Center
User Support
My Account
Billing Center
Cost Center
Resource Center
Enterprise Management
Service Tickets
HUAWEI CLOUD (International) FAQs
ICP Filing
Support Plans
My Credentials
Customer Operation Capabilities
Partner Support Plans
Professional Services
Analytics
MapReduce Service
Data Lake Insight
CloudTable Service
Cloud Search Service
Data Lake Visualization
Data Ingestion Service
GaussDB(DWS)
DataArts Studio
Data Lake Factory
DataArts Lake Formation
IoT
IoT Device Access
Others
Product Pricing Details
System Permissions
Console Quick Start
Common FAQs
Instructions for Associating with a HUAWEI CLOUD Partner
Message Center
Security & Compliance
Security Technologies and Applications
Web Application Firewall
Host Security Service
Cloud Firewall
SecMaster
Anti-DDoS Service
Data Encryption Workshop
Database Security Service
Cloud Bastion Host
Data Security Center
Cloud Certificate Manager
Edge Security
Managed Threat Detection
Blockchain
Blockchain Service
Web3 Node Engine Service
Media Services
Media Processing Center
Video On Demand
Live
SparkRTC
MetaStudio
Storage
Object Storage Service
Elastic Volume Service
Cloud Backup and Recovery
Storage Disaster Recovery Service
Scalable File Service Turbo
Scalable File Service
Volume Backup Service
Cloud Server Backup Service
Data Express Service
Dedicated Distributed Storage Service
Containers
Cloud Container Engine
SoftWare Repository for Container
Application Service Mesh
Ubiquitous Cloud Native Service
Cloud Container Instance
Databases
Relational Database Service
Document Database Service
Data Admin Service
Data Replication Service
GeminiDB
GaussDB
Distributed Database Middleware
Database and Application Migration UGO
TaurusDB
Middleware
Distributed Cache Service
API Gateway
Distributed Message Service for Kafka
Distributed Message Service for RabbitMQ
Distributed Message Service for RocketMQ
Cloud Service Engine
Multi-Site High Availability Service
EventGrid
Dedicated Cloud
Dedicated Computing Cluster
Business Applications
Workspace
ROMA Connect
Message & SMS
Domain Name Service
Edge Data Center Management
Meeting
AI
Face Recognition Service
Graph Engine Service
Content Moderation
Image Recognition
Optical Character Recognition
ModelArts
ImageSearch
Conversational Bot Service
Speech Interaction Service
Huawei HiLens
Video Intelligent Analysis Service
Developer Tools
SDK Developer Guide
API Request Signing Guide
Terraform
Koo Command Line Interface
Content Delivery & Edge Computing
Content Delivery Network
Intelligent EdgeFabric
CloudPond
Intelligent EdgeCloud
Solutions
SAP Cloud
High Performance Computing
Developer Services
ServiceStage
CodeArts
CodeArts PerfTest
CodeArts Req
CodeArts Pipeline
CodeArts Build
CodeArts Deploy
CodeArts Artifact
CodeArts TestPlan
CodeArts Check
CodeArts Repo
Cloud Application Engine
MacroVerse aPaaS
KooMessage
KooPhone
KooDrive
On this page

Show all

KafkaConsumer Monitoring

Updated on 2024-07-04 GMT+08:00

This section describes the types, names, and meanings of KafkaConsumer metrics collected by APM.

Table 1 KafkaConsumer monitoring collection parameters

Parameter

Data Type

Application Type

Default

Supported Start Agent Version

Supported End Agent Version

Description

Kafka Slow Request Threshold

integer

JAVA

800

2.1.14

-

The sampling ratio will increase if the slow request threshold is crossed.

Kafka Consumption Method Configuration

obj_array

JAVA

-

2.1.14

-

Kafka consumption method configuration

Table 2 KafkaConsumer metrics

Category

Metric

Name

Description

Unit

Data Type

Default Aggregation Mode

Topic (topic: Kafka topic monitoring data)

id

id

Client ID and IP address

-

ENUM

LAST

topic

topic

Kafka topic name

-

ENUM

LAST

bytesConsumedRate

Bytes Consumed/s

Number of bytes consumed per second

Byte

INT

AVG

fetchSizeAvg

Avg. Bytes Fetched

Average number of bytes fetched for a request

Byte

INT

AVG

fetchSizeMax

Max. Bytes Fetched

Maximum number of bytes fetched for a request

Byte

INT

MAX

recordsConsumedRate

Messages Consumed/s

Number of messages consumed per second

-

INT

AVG

recordsPerRequestAvg

Avg. Messages of Single Request

Average number of messages of a single request

-

INT

AVG

seqIds

Producer-generated SN

Sequence number generated by the producer

-

STRING

LAST

recordConsumedTotal

Total Consumption Times

Total number of consumption times

-

INT

SUM

bytesConsumedTotal

Total Consumed Bytes

Total number of bytes that have been consumed

-

INT

SUM

Fetch monitoring (fetch: Kafka fetch monitoring data)

id

id

Client ID and IP address

-

ENUM

LAST

bytesConsumedRate

Bytes Consumed/s

Number of bytes consumed per second

Byte

INT

AVG

fetchLatencyAvg

Avg. Request Latency

Average request latency

ms

INT

AVG

fetchLatencyMax

Max. Request Latency

Maximum request latency

ms

INT

MAX

fetchRate

Requests/s

Number of requests per second

-

INT

AVG

fetchSizeAvg

Avg. Bytes Fetched

Average number of bytes fetched for a request

Byte

INT

AVG

fetchSizeMax

Max. Bytes Fetched

Maximum number of bytes fetched for a request

Byte

INT

MAX

recordsConsumedRate

Messages Consumed/s

Number of messages consumed per second

-

INT

AVG

recordsLagMax

Max. Accumulated Messages

Maximum number of accumulated messages

-

INT

MAX

recordsPerRequestAvg

Avg. Messages of Single Request

Average number of messages of a single request

-

INT

AVG

seqIds

Producer-generated SN

Sequence number generated by the producer

-

STRING

LAST

recordConsumedTotal

Total Consumption Times

Total number of consumption times

-

INT

SUM

bytesConsumedTotal

Total Consumed Bytes

Total number of bytes that have been consumed

-

INT

SUM

Partition (partition: Kafka partition data)

id

id

Client ID and IP address

-

ENUM

LAST

partition

partition

Kafka partition name

-

ENUM

LAST

recordsLag

Accumulated Messages

Number of accumulated messages

-

INT

LAST

recordsLagAvg

Avg. Accumulated Messages

Average number of accumulated messages

-

INT

AVG

recordsLagMax

Max. Accumulated Messages

Maximum number of accumulated messages

-

INT

MAX

seqIds

Producer-generated SN

Sequence number generated by the producer

-

STRING

LAST

Kafka consumption method monitoring (consumer)

method

Method

Consumption method

-

ENUM

LAST

concurrentMax

Max. Concurrency

Maximum concurrency

-

INT

MAX

errorCount

Errors

Number of errors

-

INT

SUM

invokeCount

Calls

Number of calls

-

INT

SUM

lastError

Error Message

Error details

-

STRING

LAST

maxTime

Max. RT

Maximum response time in a collection period

-

INT

MAX

range1

0–10 ms

Number of requests with 0–10 ms response time

-

INT

SUM

range2

10–100 ms

Number of requests with 10–100 ms response time

-

INT

SUM

range3

100–500 ms

Number of requests with 100–500 ms response time

-

INT

SUM

range4

500–1000 ms

Number of requests with 500–1000 ms response time

-

INT

SUM

range5

1–10s

Number of requests with 1–10s response time

-

INT

SUM

range6

> 10s

Number of requests with more than 10s response time

-

INT

SUM

totalTime

Total RT

Total response time

-

INT

SUM

KafkaConsumer summary (total)

recordConsumedTotal

Total Consumption Times

Total number of consumption times

-

INT

SUM

bytesConsumedTotal

Total Consumed Bytes

Total number of bytes that have been consumed

-

INT

SUM

recordsLag

Total Accumulated Messages

Total number of messages that have been accumulated

-

INT

LAST

Exception (exception: exception statistics about Kafka consumption)

causeType

Exception Class

Exception class

-

ENUM

LAST

exceptionType

Exception Class

Exception class

-

ENUM

LAST

count

Count

Number of exceptions

-

INT

SUM

message

Exception Message

Exception message

-

STRING

LAST

stackTrace

Exception Stack

Exception stack

-

CLOB

LAST

We use cookies to improve our site and your experience. By continuing to browse our site you accept our cookie policy. Find out more

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback