Help Center/
ModelArts/
Troubleshooting/
Training Jobs/
In-Cloud Migration Adaptation Issues/
Error Message "retCode=0x91, [the model stream execute failed]" Displayed in MindSpore Logs
Updated on 2025-08-22 GMT+08:00
Error Message "retCode=0x91, [the model stream execute failed]" Displayed in MindSpore Logs
Symptom
When MindSpore is used for training, the following error message is displayed:
[ERROR] RUNTIME(3002)model execute error, retCode=0x91, [the model stream execute failed]
Possible Causes
The possible causes are as follows:
The speed of reading data cannot keep up with the model iteration speed.
Solution
- Reduce shuffle operations during preprocessing.
dataset = dataset.shuffle(buffer_size=x)
- Disable data preprocessing, which may affect system performance.
NPURunConfig(enable_data_pre_proc=false)
Summary and Suggestions
Before creating a training job, use the ModelArts development environment to debug your training code and minimize migration errors.
- Use the notebook environment for online debugging. For details, see Using JupyterLab to Develop Models.
- Use a local IDE (PyCharm or VS Code) to access the cloud environment for debugging. For details, see Using a Local IDE to Develop Models.
Parent topic: In-Cloud Migration Adaptation Issues
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
The system is busy. Please try again later.
For any further questions, feel free to contact us through the chatbot.
Chatbot