Doris Logs

Updated on 2024-11-29 GMT+08:00

View PDF

Description

Log path: Doris logs are stored in /var/log/Bigdata/doris/role name by default.

FE: /var/log/Bigdata/doris/fe (run logs) and /var/log/Bigdata/audit/doris/fe (audit logs)
BE: /var/log/Bigdata/doris/be (run logs)
DBroker: /var/log/Bigdata/doris/dbroker (run logs)

Log archive rule: The automatic compression and archive function is enabled for Doris logs. By default, when a log file exceeds a specified size (which is configurable), the log file is automatically compressed. The naming rule of the compressed log file is as follows: <Original log file name>-<yyyy-mm-dd_hh-mm-ss>.[ID].log.zip. A maximum of 20 latest compressed files are retained by default. The number of compressed files and compression threshold can be changed.

**Table 1** Doris logs
Log Type	Log File	Description
Run log	/fe/fe.out	Standard/Error logs (stdout and stderr)
	/fe/fe.log	Main log, including all contents except fe.out
	/fe/fe.warn.log	Subset of fe.log. Only WARN and ERROR logs are recorded.
	/fe/fe-omm-<Date>-<PID>-gc.log.<No.>	GC logs of the FE process
	/fe/preStart.log	Work logs before the FE starts
	/fe/check_fe_status.log.log	Log file that records whether the FE service is started successfully
	/fe/cleanup.log	Cleanup log for FE uninstallation
	/fe/start_fe.log	FE process startup log
	/fe/stop_fe.log	FE process stop log
	/fe/postinstallDetail.log	Work logs generated after the FE is installed and before it starts
	/be/be.INFO	Run log of the BE process
	be.WARNING	Subset of be.log. Only WARN and FATAL logs are recorded.
	/be/be-omm-<Date>-<PID>-gc.log.<No.>	GC logs of the BE process
	/be/postinstallDetail.log	Work logs generated after BE is installed and before it starts
	/be/preStart.log	Work logs before BE starts
	/be/cleanup.log	Cleanup log for BE uninstallation
	/be/start_be.log	BE process startup log
	/be/stop_be.log	BE process stop log
	/be/check_be_status.log	Log file that records whether the BE service is started successfully
	/be/be.out	Standard/Error output logs of the BE process (stdout and stderr)
	/dbroker/start_broker.log	Log file that records the normal start and stop of the DBroker process
	/dbroker/stop_broker.log	log file that records start and stop exceptions of the DBroker process
	/dbroker/preStart.log	Work log before DBroker starts
	/dbroker/cleanup.log	Cleanup log generated during or before DBroker uninstallation
	/dbroker/check_db_status.log	Log file that records whether the DBroker service is started successfully
	/dbroker/dbroker-omm-<Date>-<PID>-gc.log.<No.>	GC log of the DBroker process
	/dbroker/apache_hdfs_broker.log	Run log of the DBroker process
Audit log	fe.audit.log	Audit log, which records all SQL requests received by the FE

Log Levels

Table 2 describes the log levels supported by Doris.

The priorities of run log levels are FATAL, ERROR, WARN, and INFO in descending order. Logs whose levels are higher than or equal to a specified level are displayed. The number of displayed logs decreases as the specified log level increases.

**Table 2** Log levels
Level	Description
FATAL	Logs of this level record program assertion errors
ERROR	Logs of this level record error information about system running
WARN	Logs of this level record exception information about the current event processing
INFO	Logs of this level record normal running status information about the system and events

To change log levels, perform the following operations:

Log in to FusionInsight Manager and choose Cluster > Services > Doris > Configurations > All Configurations. The All Configurations page of the Doris service is displayed.
On the menu bar on the left, select the log menu of the target role.
Select a desired log level and save the configuration.

NOTE:

The Doris log level takes effect immediately after being configured. You do not need to restart the service.

Log Formats

The following table describes Doris log formats and gives you some examples:

**Table 3** Log format
Log Type	Format	Example
FE run log	<yyyy-MM-dd HH:mm:ss,SSS><LogLevel>(Thread name\|Thread ID)<Location where the log event occurs>\|<Messages in the log>	2023-04-13 11:17:14,371 INFO (tablet stat mgr\|34) [TabletStatMgr.runAfterCatalogReady():125] finished to update index row num of all databases. cost: 0 ms
BE run log	<Log level. I for INFO, W for WARN, F for FATAL\|MMdd HH:mm:ss.SSS> <Thread ID> <Location where a log event occurs> <Messages in the log>	I0413 11:26:03.439189 25248 tablet_manager.cpp:895] begin to build all report tablets info
DBroker run log	<MMdd HH:mm:ss.SSS> <Thread ID> <Log Level><Messages in the log>	2023-04-11 11:43:13 [ main:0 ] - [ INFO ] starting apache hdfs broker....
Audit log	<yyyy-MM-dd HH:mm:ss,SSS[Operation type] >\|<Client>\|<User Name>\|<Db Name>\|<State>\|<ErrorCode>\|<ErrorMessage>\|<Time>\|<ScanBytes>\|<ScanRows>\|<ReturnRows>\|<StmtId>\|<QueryId>\|<IsQuery>\|<feIp>\| <Stmt>\|<CpuTimeMS>\|<SqlHash>\|<peakMemoryBytes>\|<SqlDigest>\|<TraceId>\|<FuzzyVariables>	2023-04-13 10:49:26,410 [query] \|Client=192.168.64.223:44382\|User=root\|Db=hivedoris\|State=ERR\|ErrorCode=1105\|ErrorMessage=errCode = 2, detailMessage = (192.168.64.78)[INTERNAL_ERROR]failed to init reader for file /user/hive/warehouse/hivedoris.db/test/000000_0, err: [INTERNAL_ERROR]connect to hdfs failed. error: (255), Unknown error 255), reason: NullPointerException: \|Time=67\|ScanBytes=0\|ScanRows=0\|ReturnRows=0\|StmtId=91\|QueryId=e1125283f12c4994-a69e3a323044d681\|IsQuery=true\|feIp=192.168.64.78\|Stmt=select * from test\|CpuTimeMS=0\|SqlHash=3bbc220823c3e757002fb9490196cf84\|peakMemoryBytes=0\|SqlDigest=\|TraceId=\|FuzzyVariables=