Help Center/ Log Tank Service/ User Guide/ Log Ingestion/ Self-Built Middleware (Beta)/ ZooKeeper Logs
Updated on 2025-03-28 GMT+08:00
View PDF
Share

ZooKeeper Logs

ZooKeeper is a distributed, open-source coordination service for distributed applications. It encapsulates complex and error-prone distributed consistency services into an efficient, reliable system, enabling a series of simple and easy-to-use atomic operations. ZooKeeper's main functions include configuration maintenance, domain name services, distributed synchronization, and group services. It provides a distributed exclusive lock, election, and queue interface for consistency, availability, and concurrency control in a distributed system.

ZooKeeper logs are classified into the following three types:

  1. Transaction logs:
    • These logs record all transaction operations in ZooKeeper clusters, including requests from clients and responses from servers.
    • The default storage location of transaction logs is specified by the dataLogDir parameter in the zoo.cfg configuration file. If dataLogDir is not configured, transaction logs are stored in the directory specified by dataDir.
    • The transaction log directory includes a subdirectory named version-2. version indicates the version of the transaction log format used by ZooKeeper. This subdirectory includes a series of files named log.<txid>, where <txid> is a hexadecimal number indicating the ID of the first transaction that writes the log.
    • Transaction logs are binary files and cannot be viewed with a text editor. You can use the tool provided by ZooKeeper (for example, the printlog command in the zkServer.sh script) to view and parse transaction logs.
  2. Snapshot logs:
    • A snapshot log is a serialized representation of the ZooKeeper memory data status. It contains a complete backup of the ZooKeeper data tree at a certain point in time.
    • Snapshot logs are stored in the directory specified by dataLogDir or dataDir. The snapshot log directory includes a subdirectory named version-2 and a series of files named snapshot.<zxid>, where <zxid> is a hexadecimal number indicating the ID of the last transaction committed when the snapshot is triggered.
    • Different from transaction logs, snapshot logs are text files and can be viewed using a text editor. Since snapshot logs contain a large amount of data, they may be very large.
  3. System logs:

    ZooKeeper system logs (also called run logs or application logs) record various information generated during the ZooKeeper cluster running, including startup and stop information, configuration loading, node connection and disconnection, request processing, and election process.

Collection Method

Install ICAgent and configure ECS text log ingestion by referring to Installing ICAgent (Intra-Region Hosts) and Ingesting ECS Text Logs to LTS. Then, ICAgent will collect ECS text logs and report them to LTS.

ZooKeeper Log Examples and Fields

2024-05-06 10:28:25,123 [myThread-1] INFO org.apache.zookeeper.server.ZooKeeperServer - Accepted socket connection from /192.168.1.100:56789
Table 1 ZooKeeper log fields

Field

Description

Example

Timestamp

Time when the log was generated, in the format of yyyy-MM-dd HH:mm:ss,SSS.

2024-05-06 10:28:25,123

Thread ID

ID of the thread that generated the log.

myThread-1

Log level

Common log levels include DEBUG, INFO, WARN, and ERROR.

INFO

Log source

The log was generated by the org.apache.zookeeper.server.ZooKeeperServer class.

org.apache.zookeeper.server.ZooKeeperServer

Log content

-

Accepted socket connection from /192.168.1.100:56789: The ZooKeeper server accepted a socket connection from the IP address 192.168.1.100 and port number 56789.

Log Location

The location of ZooKeeper system log files (also called run logs or application logs) depends on the ZooKeeper configuration. You can specify it by configuring the log4j.properties file. For example, set zookeeper.log.dir in the file. You can also find a configuration item named zookeeper.log.dir in the file. It specifies the directory of log files. Example:

zookeeper.log.dir=/path/to/zookeeper/logs