HetuEngine Logs

Updated on 2024-11-29 GMT+08:00

View PDF

Log Description

Log paths:

The HetuEngine logs are stored in /var/log/Bigdata/hetuengine/ and /var/log/Bigdata/audit/hetuengine/.

Log archiving rules:

Log archiving rules use the FixedWindowRollingPolicy policy. The maximum size of a single file and the maximum number of log archive files can be configured. The rules are as follows:

When the size of a single file exceeds the default maximum value, a new compressed archive file is generated. The naming rule of the compressed archive log file is as follows: <Original log name>.[ID].log.gz.
When the number of log archive files reaches the maximum value, the earliest log file is deleted.

By default, the maximum size of an audit log file is 30 MB, and the maximum number of log archive files is 20.

By default, the maximum size of a run log file is 100 MB, and the maximum number of log archive files is 20.

To change the maximum size of a single run log file or audit log file or change the maximum number of log archive files of an instance, perform the following operations:

Log in to Manager.
Choose Cluster > Services > HetuEngine > Configurations > All Configurations.
In the parameter list of log levels, search for logback.xml to view the current run log and audit log configurations of HSBroker, HSConsole, HSFabric, and QAS.
Select the configuration item to be modified and modify it.
Click Save, and then click OK. The configuration automatically takes effect after about 30 seconds.

**Table 1** HetuEngine log list
Log Category	Log File	Description
Installation, startup, and stop log	prestart.log	Preprocessing script log before startup
	start.log	Startup log
	stop.log	Stop log
	postinstall.log	Installation log
Run log	Instance name.log	Run log
	Instance name_wsf.log	Interface parameter verification log
	hdfs://hacluster/hetuserverhistory/Tenant/Coordinator or worker/application_ID/container_ID/yyyyMMdd/server.log	Run log of the HetuEngine compute instance
Status check log	service_check.log	Health check log
	service_getstate.log	Status check log
	availability-check.log	HetuEngine status check log
	haCheck.log	Log generated when the QAS checks the HA status
Audit log	Instance name-audit.log	Audit log
	hsbroker-audit.log	Audit log of HSBroker operations
	hsconsole-audit.log	Audit log of HSConsole operations
	hsfabric-audit.log	Audit log of HetuEngine operations performed across domains
	hdfs://hacluster/hetuserverhistory/Tenant/coordinator/application_ID/container_ID/yyyyMMdd/hetuserver-engine-audit.log	Audit log of the HetuEngine compute instance
queryInfo log	hdfs://hacluster/hetuserverhistory/Tenant/Coordinator/application_ID/container_ID/yyyyMMdd/queryinfo.log	queryInfo log of HetuEngine compute instances, which records SQL running statistics.
Clean log	cleanup.log	Script cleanup log
Initialization log	hetupg.log	Metadata initialization log
Initialization log	ranger-trino-plugin-enable.log	Operation log of integrating Ranger plug-ins to the HetuEngine kernel
Client log	qas_client.log	ZooKeeper client log of the QAS instance
Stack information log	threadDump-<DATE>.log	Log printed when instances are restarted or stopped
Other	hetu-updateKrb5.log	Log generated when the Hive data source configuration is automatically updated after the Hive cluster domain is changed.
Other	hetu_utils.log	Log generated when the preprocessing script calls the tool class to upload files to the HDFS during startup.

Log Level

Table 2 describes the log levels provided by HetuEngine. The priorities of log levels are OFF, ERROR, WARN, INFO, and DEBUG in descending order. Logs whose levels are higher than or equal to the specified level are printed. The number of printed logs decreases as the specified log level increases.

**Table 2** Log levels
Level	Description
OFF	Logs of this level record no logs.
ERROR	Logs of this level record error information about the current event processing.
WARN	Logs of this level record exception information about the current event processing.
INFO	Logs of this level record normal running status information about the system and events.
DEBUG	Logs of this level record the system information and system debugging information.

To change the run log or audit log level of an instance, perform the following steps:

Log in to FusionInsight Manager.
Choose Cluster > Services > HetuEngine > Configurations > All Configurations.
In the parameter list of log levels, search for logback.xml to view the current run log and audit log levels of HSBroker, HSConsole, and HSFabric.
Select a desired log level.
Click Save, and then click OK. The configuration automatically takes effect after about 30 seconds.

To change the HetuEngine Coordinator/Worker log level, perform the following steps:

Log in to FusionInsight Manager.
Choose Cluster > Services > HetuEngine > Configurations > All Configurations.
In the parameter list of log levels, search for log.properties to view the current log levels.
Select a desired log level.
Click Save, and then click OK. Wait until the operation is successful.
Choose Cluster > Services > HetuEngine > Instance, click the HSBroker instance in the role list, and choose More > Restart Instance.
After the HSBroker instance is restarted, choose Cluster > Services > HetuEngine. On the overview page, click the link next to HSConsole WebUI to go to the compute instance page.
Select the compute instances to be restarted and click Stop. After all instances are stopped, click Start to restart the compute instances.

Parent topic: Using HetuEngine

Previous topic: HetuEngine UDF Development and Application

Next topic: HetuEngine Performance Tuning

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot