Help Center> ModelArts> Troubleshooting> Training Jobs> Service Code Issues> Error Message "CUDNN_STATUS_NOT_SUPPORTED" Displayed in Logs
Updated on 2022-12-08 GMT+08:00

Error Message "CUDNN_STATUS_NOT_SUPPORTED" Displayed in Logs

Symptom

The following error message is displayed during PyTorch training:
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_SUPPORTED. This error may appear if you passed in a non-contiguous input.

Possible Causes

The input data is not of contiguous type, which is not supported by cuDNN.

Solution

  1. Disable cuDNN before training.
    torch.backends.cudnn.enabled = False
  2. Convert the input data into contiguous data.
    images = images.cuda()  
    images = images.permute(0, 3, 1, 2).contigous()

Summary and Suggestions

Before creating a training job, use the ModelArts development environment to debug the training code to maximally eliminate errors in code migration.