Updated on 2024-10-09 GMT+08:00

CDL Log Overview

Log Description

Log path: The default log storage path of CDL is /var/log/Bigdata/cdl/Role name abbreviation.

  • CDLService: /var/log/Bigdata/cdl/service (run logs) and /var/log/Bigdata/audit/cdl/service (audit logs).
  • CDLConnector: /var/log/Bigdata/cdl/connector (run logs).

Log archive rule: The automatic CDL log compression function is enabled. By default, when the size of logs exceeds 50 MB, logs are automatically compressed into a log file named in the following format: <Original log name>.<yyyy-mm-dd_hh-mm-ss>.[ID].zip. A maximum of 20 latest compressed files are reserved by default. The number of compressed files can be configured on Manager.

Table 1 Log list

Type

File

Description

Run logs

connect.log

CDLConnector run log.

prestartDetail.log

Log that records cluster initialization before service startup.

startDetail.log

Service startup log.

stopDetail.log

Service stop log.

cleanupDetail.log

Log that records the cleanup execution of services.

check-serviceDetail.log

Log that records the verification of service status after service installation.

cdl-db-operation.log

Log that records database initialization during service startup.

cdl-app-launcher.log

Spark application startup log of CDL data synchronization tasks.

cdl-dc-app-launcher.log

Spark application startup log of CDL data comparison tasks.

serviceInstanceCheck.log

Instance check log of CDLService.

connectorInstanceCheck.log

Instance check log of CDLConnector.

ModifyDBPasswd.log

Log that records the resetting of the service database password.

ranger-cdl-plugin-enable.log

Log that records the enabling or disabling of Ranger authentication.

postinstallDetail.log

Service installation log.

cdl_connector_pidxxx_gc.log.x

CDLConnector garbage collection (GC) log.

cdl_service_pidxxx_gc.log.x

CDLService GC log.

threadDump-CDLConnector-xxx.log

CDLConnector stack log.

threadDump-CDLService-xxx.log

CDLService stack log.

Audit log

cdl-audit.log

Service audit log.

Log Level

Table 2 describes the log levels supported by CDL.

Levels of run logs are FATAL, ERROR, WARN, INFO, and DEBUG from the highest to the lowest priority. Run logs of equal or higher levels are recorded. The higher the specified log level, the fewer the logs recorded.

Table 2 Log levels

Type

Level

Description

Run log and audit log

FATAL

Logs of this level record fatal information about system.

ERROR

Logs of this level record error information about system running.

WARN

Logs of this level record exception information about the current event processing.

INFO

Logs of this level record normal running status information about the system and events.

DEBUG

Logs of this level record system running and debugging information.

To modify log levels, perform the following operations:

  1. Go to the All Configurations page of CDL. For details, see Modifying Cluster Service Configuration Parameters.
  2. On the menu bar on the left, select the log menu of the target role.
  3. Select a desired log level.
  4. Save the configuration. In the displayed dialog box, click OK to make the configurations take effect.

    The configurations take effect immediately without the need to restart the service.

Log Format

The following table lists the CDL log formats:

Table 3 Log formats

Type

Format

Example

Run log

<yyyy-MM-dd HH:mm:ss,SSS>|<Log level>|<Name of the thread that generates the log>|<Message in the log>|<Location where the log event occurs>

2021-06-15 17:25:19,658 | DEBUG | qtp2009591182-1754 | >fill SslConnection@5d04c5a0::SocketChannelEndPoint@7c011c24{l=/10.244.224.65:21495,r=/10.244.224.83:53724,OPEN,fill=-,flush=-,to=1/30000}{io=0/0,kio=0,kro=1}->SslConnection@5d04c5a0{NOT_HANDSHAKING,eio=-1/-1,di=-1,fill=IDLE,flush=IDLE}~>DecryptedEndPoint@771f2f77{l=/10.244.224.65:21495,r=/10.244.224.83:53724,OPEN,fill=-,flush=-,to=19398/30000}=>HttpConnection@68c5859b[p=HttpParser{s=CONTENT,0 of -1},g=HttpGenerator@536e2de0{s=END}]=>HttpChannelOverHttp@7bf252bd{s=HttpChannelState@38be31e{s=IDLE rs=COMPLETED os=COMPLETED is=IDLE awp=false se=false i=false al=0},r=1,c=true/true,a=IDLE,uri=https://10.244.224.65:21495/api/v1/cdl/monitor/jobs/metrics,age=19382} | SslConnection.java:614

Audit log

<yyyy-MM-dd HH:mm:ss,SSS>|<Log level>|<Name of the thread that generates the log>|<Message in the log>|<Location where the log event occurs>

2021-06-15 11:07:00,262 | INFO | qtp1846345504-30 | STARTTIME=2021-06-15 11:06:47.912 ENDTIME=2021-06-15 11:07:00.261 USERIP=10.144.116.198 USER=CDL User INSTANCE=10-244-224-65 OPERATION=Start CDL Job TARGET=CDCJobExecutionResource RESULT=SUCCESS | CDCAuditLogger.java:93