System Container Exits Unexpectedly
Symptom
After a training job is created, the system container exits unexpectedly.
[ModelArts Service Log]2022-10-11 19:18:23,267 - file_io.py[1ine:748] - ERROR: stat:404 errorCode:NoSuchKey errorMessage:The specifiedkey does not exist. reason:Not Found request-id:00000183C6C4010C66D399E000COE3xx retry:0 [ModelArts Service Log]2022-10-11 19:18:23,267 - modelarts-downloader.py[line:90] - ERROR: modelarts-downloader. py: Download directory failed: [Errno {'status': 404, ......}] file or directoryor bucket not found.
Possible Causes
The possible causes are as follows:
- An error occurred in OBS.
- Unavailable file: The specified key does not exist.
- Insufficient OBS permissions
- OBS traffic limiting
- Others
- The disk space is insufficient.
Solution
- For an OBS error:
- Unavailable file: The specified key does not exist.
For details, see Error Message "errorMessage:The specified key does not exist" Displayed in Logs.
- Insufficient OBS permissions
For details, see Error "stat:403 reason:Forbidden" Is Displayed in Logs When a Training Job Accesses OBS.
- OBS traffic limiting
For details, see Error Message "BrokenPipeError: Broken pipe" Is Displayed When OBS Data Is Copied.
- Others
For details, see OBS Server-Side Error Codes. Alternatively, obtain the request ID and contact OBS customer service.
- Unavailable file: The specified key does not exist.
- For insufficient disk space:
For details, see Insufficient Disk Space.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot