Help Center/ ModelArts/ Model Development/ Performing a Training/ Training Job Logs/ Common Logs
Updated on 2023-09-20 GMT+08:00
View PDF
Share

Common Logs

Common logs include the logs for pip-requirement.txt, training process, and ModelArts.

Log Type

Table 1 Log type

Type

Description

Training process log

Standard output of your training code

Installation logs for pip-requirement.txt

If pip-requirement.txt is defined in training code, PIP package installation logs are generated.

ModelArts logs

ModelArts logs are used by O&M personnel to locate service faults.

File Format

The format of a common log file is as follows. task id is the node ID of a training job.

Unified log format: modelarts-job-[job id]-[task id].log
Example: log/modelarts-job-95f661bd-1527-41b8-971c-eca55e513254-worker-0.log
  • Single-node training jobs generate a log file, and task id defaults to worker-0.
  • Distributed training generates multiple node log files, which are distinguished by task id, such as worker-0 and worker-1.

Common logs include the logs for pip-requirement.txt, training process, and ModelArts.

ModelArts Logs

ModelArts logs can be filtered in the common log file modelarts-job-[job id]-[task id].log using the following keywords: [ModelArts Service Log] or Platform=ModelArts-Service.

  • Type 1: [ModelArts Service Log] xxx 
    [ModelArts Service Log][init] download code_url: s3://dgg-test-user/snt9-test-cases/mindspore/lenet/
  • Type 2: time="xxx" level="xxx" msg="xxx" file="xxx" Command=xxx Component=xxx Platform=xxx 
    time="2021-07-26T19:24:11+08:00" level=info msg="start the periodic upload task, upload period = 5 seconds " file="upload.go:46" Command=obs/upload Component=ma-training-toolkit Platform=ModelArts-Service
Parent topic: Training Job Logs