Flink Log Overview

Log Description

Log archive rules:

1. FlinkResource run logs:
   • By default, service logs are backed up each time when the log size reaches 20 MB. A maximum of 20 logs can be reserved without being compressed.
     

     For versions earlier than MRS 3.x, The executor logs are backed up each time when the log size reaches 30 MB. A maximum of 20 logs can be reserved without being compressed.

   • You can set the log size and number of compressed logs on the Manager page or modify the corresponding configuration items in log4j-cli.properties, log4j.properties, and log4j-session.properties in Client installation directory/Flink/flink/conf/ on the client.

   Table 1 FlinkResource log list

   Type	Name	Description
   FlinkResource run logs	checkService.log	Health check log
   	kinit.log	Initialization log
   	postinstall.log	Service installation log
   	prestart.log	Prestart script log
   	start.log	Startup log

2. FlinkServer service logs and audit logs.
   • By default, FlinkServer service logs and audit logs are backed up each time when the log size reaches 100 MB. The service logs are stored for a maximum of 30 days, and audit logs are stored for a maximum of 90 days.
   • You can set the log size and number of compressed logs on the Manager page or modify the corresponding configuration items in log4j-cli.properties, log4j.properties, and log4j-session.properties in Client installation directory/Flink/flink/conf/ on the client.

   Table 2 FlinkServer log list

   Type	Name	Description
   FlinkServer run logs	checkService.log	Health check log
   	checkFlinkServer.log	Health check log of FlinkServer
   	localhost_access_log..yyyy-mm-dd.txt	URL log of FlinkServer
   	start_thrift_server.out	Thrift server startup log
   	thrift_server_thriftServer_xxx.log.last
   	cleanup.log	Cleanup log file for instance installation and uninstallation
   	flink-omm-client-IP.log	Job startup log
   	flinkserver_yyyymmdd-x.log.gz	Service archive log
   	flinkserver.log	Service log
   	flinkserver---pidxxxx-gc.log.x.current	GC log
   	kinit.log	Initialization log
   	postinstall.log	Service installation log
   	prestart.log	Prestart script log
   	start.log	Startup log
   	stop.log	Stop log
   	catalina.yyyy-mm-dd.log	Tomcat run log
   	catalina.out
   	host-manager.yyyy-mm-dd.log
   	localhost.yyyy-mm-dd.log
   	manager.yyyy-mm-dd.log
   FlinkServer audit logs	flinkserver_audit_yyyymmdd-x.log.gz	Audit archive log
   	flinkserver_audit.log	Audit log

Log Level

Table 3 describes the log levels supported by Flink. The priorities of log levels are ERROR, WARN, INFO, and DEBUG in descending order. Logs whose levels are higher than or equal to the specified level are printed. The number of printed logs decreases as the specified log level increases.

To modify log levels, perform the following steps:

1. Go to the All Configurations page of Flink by referring to Modifying Cluster Service Configuration Parameters.
2. On the menu bar on the left, select the log menu of the target role.
3. Select a desired log level.
4. Save the configuration. In the displayed dialog box, click OK to make the configurations take effect.

• After the configuration is complete, you do not need to restart the service. Download the client again for the configuration to take effect.
• You can also change the configuration items corresponding to the log level in log4j-cli.properties, log4j.properties, and log4j-session.properties in Client installation directory/Flink/flink/conf/ on the client.
• When a job is submitted using a client, a log file is generated in the log folder on the client. The default umask value is 0022. Therefore, the default log permission is 644. To change the file permission, you need to change the umask value. For example, to change the umask value of user omm:
  • Add umask 0026 to the end of the /home/omm/.baskrc file.
  • Run the source /home/omm/.baskrc command to make the file permission take effect.

Log Format