Kafka Logs

Collection Method

Install ICAgent and configure ECS text log ingestion by referring to Installing ICAgent (Intra-Region Hosts) and Ingesting ECS Text Logs to LTS. Then, ICAgent will collect ECS text logs and report them to LTS.

Kafka Log Examples and Fields

Kafka does not directly provide the detailed format or field definition of system logs. The format or field definition may vary according to the log framework. This section uses a typical log framework, Log4j, as an example to describe some common fields.

1. System log example

   2023-10-23 15:33:45,123 INFO [KafkaServer id=0] onBrokerStartup (KafkaServer.scala:220) starting  
   2023-10-23 15:33:45,567 WARN [ReplicaManager broker=0] topic=my_topic partition=0 Broker 1 disconnected; no more in-sync replicas  
   2023-10-23 15:33:46,987 ERROR [Controller id=0] onControllerFailover (KafkaController.scala:678) Error during controller failover  
   java.lang.RuntimeException: Failed to elect new leader for partition [my_topic,0]  
       at kafka.controller.KafkaController
   $$
   anonfun$onControllerFailover$1
   $$
   anonfun$apply$mcV$sp$1.apply$mcV$sp(KafkaController.scala:629)  
       at kafka.controller.KafkaController
   $$
   anonfun$onControllerFailover$1
   $$
   anonfun$apply$mcV$sp$1.apply(KafkaController.scala:609)  
       at kafka.controller.KafkaController
   $$
   anonfun$onControllerFailover$1
   $$
   anonfun$apply$mcV$sp$1.apply(KafkaController.scala:609)  
       ... (stack trace continues)

2. Field description

   In the preceding example, the first is an informational log entry generated when KafkaServer was started. The second is a warning log entry, indicating that an in-sync replica of a partition was disconnected from the broker. The third is an error log entry, indicating that an error occurred during controller failover and an attempt to select a new leader for a partition failed.

   Table 1 Fields

   Field	Description	Example
   Timestamp	Time when the log entry was generated.	2023-10-23 15:33:45,123
   Log level	Importance or urgency of the log entry, for example, INFO, WARN, or ERROR.	INFO
   Thread or process information	Context for generating logs, such as the ID of KafkaServer or the broker ID of ReplicaManager.	[KafkaServer id=0]
   Log source	Source function and code location of the log entry.	onBrokerStartup (KafkaServer.scala:220)
   Log content	Log details, such as descriptive text, error messages, and stack traces.	starting

Log Location

By default, Kafka system log files are stored in the logs subdirectory of the Kafka installation directory. The specific location may vary depending on your configuration. The following are common methods for locating Kafka system log files:

1. Check the Kafka installation directory.

   Generally, the system log files are stored in a subdirectory named logs in the installation directory. For example, if Kafka is installed in /usr/local/kafka, log files may be stored in /usr/local/kafka/logs.

2. Check the Kafka configuration file.

   Change the logs directory of Kafka in ${KAFKA_HOME}/bin/kafka-run-class.sh.

   # Log directory to use
   if [ "x$LOG_DIR" = "x" ]; then
       # LOG_DIR="$base_dir/logs"
       LOG_DIR="/data/kafka_logs"
   fi