ZooKeeper Logs
ZooKeeper is a distributed, open-source coordination service for distributed applications. It encapsulates complex and error-prone distributed consistency services into an efficient, reliable system, enabling a series of simple and easy-to-use atomic operations. ZooKeeper's main functions include configuration maintenance, domain name services, distributed synchronization, and group services. It provides a distributed exclusive lock, election, and queue interface for consistency, availability, and concurrency control in a distributed system.
ZooKeeper logs are classified into the following three types:
- Transaction logs:
- These logs record all transaction operations in ZooKeeper clusters, including requests from clients and responses from servers.
- The default storage location of transaction logs is specified by the dataLogDir parameter in the zoo.cfg configuration file. If dataLogDir is not configured, transaction logs are stored in the directory specified by dataDir.
- The transaction log directory includes a subdirectory named version-2. version indicates the version of the transaction log format used by ZooKeeper. This subdirectory includes a series of files named log.<txid>, where <txid> is a hexadecimal number indicating the ID of the first transaction that writes the log.
- Transaction logs are binary files and cannot be viewed with a text editor. You can use the tool provided by ZooKeeper (for example, the printlog command in the zkServer.sh script) to view and parse transaction logs.
- Snapshot logs:
- A snapshot log is a serialized representation of the ZooKeeper memory data status. It contains a complete backup of the ZooKeeper data tree at a certain point in time.
- Snapshot logs are stored in the directory specified by dataLogDir or dataDir. The snapshot log directory includes a subdirectory named version-2 and a series of files named snapshot.<zxid>, where <zxid> is a hexadecimal number indicating the ID of the last transaction committed when the snapshot is triggered.
- Different from transaction logs, snapshot logs are text files and can be viewed using a text editor. Since snapshot logs contain a large amount of data, they may be very large.
- System logs:
ZooKeeper system logs (also called run logs or application logs) record various information generated during the ZooKeeper cluster running, including startup and stop information, configuration loading, node connection and disconnection, request processing, and election process.
Collection Method
Install ICAgent and configure ECS text log ingestion by referring to Installing ICAgent (Intra-Region Hosts) and Ingesting ECS Text Logs to LTS. Then, ICAgent will collect ECS text logs and report them to LTS.
ZooKeeper Log Examples and Fields
2024-05-06 10:28:25,123 [myThread-1] INFO org.apache.zookeeper.server.ZooKeeperServer - Accepted socket connection from /192.168.1.100:56789
Field |
Description |
Example |
---|---|---|
Timestamp |
Time when the log was generated, in the format of yyyy-MM-dd HH:mm:ss,SSS. |
2024-05-06 10:28:25,123 |
Thread ID |
ID of the thread that generated the log. |
myThread-1 |
Log level |
Common log levels include DEBUG, INFO, WARN, and ERROR. |
INFO |
Log source |
The log was generated by the org.apache.zookeeper.server.ZooKeeperServer class. |
org.apache.zookeeper.server.ZooKeeperServer |
Log content |
- |
Accepted socket connection from /192.168.1.100:56789: The ZooKeeper server accepted a socket connection from the IP address 192.168.1.100 and port number 56789. |
Log Location
The location of ZooKeeper system log files (also called run logs or application logs) depends on the ZooKeeper configuration. You can specify it by configuring the log4j.properties file. For example, set zookeeper.log.dir in the file. You can also find a configuration item named zookeeper.log.dir in the file. It specifies the directory of log files. Example:
zookeeper.log.dir=/path/to/zookeeper/logs
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot