Updated on 2025-07-30 GMT+08:00

What Do I Do If the Driver Is Unavailable After the Kernel Is Upgraded?

Symptom

  • When you run the nvidia-smi command, the error message "failed to initialize NVML: Driver/library version mismatch" is displayed.
  • When you run the nvidia-smi command, the error message "NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA diver" is displayed.

Checking the Kernel Version

  1. Run the following command to check the current kernel version:

    uname -r

  2. Run the corresponding command on the server to check the version of the kernel when the driver was installed:
    • CentOS: find /usr/lib/modules -name nvidia.ko
    • Ubuntu: find /lib/modules -name nvidia.ko

    If the current kernel version is different from the kernel version when the driver was installed, the driver is unavailable after the kernel is upgraded.

Solution

  1. Run the following commands in sequence to remove the NVIDIA driver:

    rmmod nvidia_drm

    rmmod nvidia_modeset

    rmmod nvidia

  2. Run the following command to query the GPU information:

    nvidia-smi