Help Center/
ModelArts/
Troubleshooting/
Training Jobs/
In-Cloud Migration Adaptation Issues/
Error Message "retCode=0x91, [the model stream execute failed]" Displayed in MindSpore Logs
Updated on 2024-06-11 GMT+08:00
Error Message "retCode=0x91, [the model stream execute failed]" Displayed in MindSpore Logs
Symptom
When MindSpore is used for training, the following error message is displayed:
[ERROR] RUNTIME(3002)model execute error, retCode=0x91, [the model stream execute failed]
Possible Causes
The speed of reading data cannot keep up with the model iteration speed.
Solution
- Reduce shuffle operations during preprocessing.
dataset = dataset.shuffle(buffer_size=x)
- Disable data preprocessing, which may affect system performance.
NPURunConfig(enable_data_pre_proc=Flase)
Summary and Suggestions
Before creating a training job, use the ModelArts development environment to debug the training code to maximally eliminate errors in code migration.
Parent topic: In-Cloud Migration Adaptation Issues
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.
The system is busy. Please try again later.
