OSError: [WinError 126] 找不到指定的模块。 Error loading “......caffe2_nvrtc.dll“ or one of its dependencies.

Posted 2023-03-04 Alex_996

昨天晚上跑的好好的训练脚本，今天早晨突然就开始报错了。主要是在Jupyter Notebook上跑的时候报错，但是直接打开Python Console跑是可以的。

Jupyter Notebook

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
~\\AppData\\Local\\Temp\\ipykernel_19784\\718949191.py in <module>
----> 1 import torch as th
      2 import pytorch_lightning as pl

D:\\anaconda3\\lib\\site-packages\\torch\\__init__.py in <module>
    137                 err = ctypes.WinError(ctypes.get_last_error())
    138                 err.strerror += f' Error loading "dll" or one of its dependencies.'
--> 139                 raise err
    140 
    141     kernel32.SetErrorMode(prev_error_mode)

OSError: [WinError 126] 找不到指定的模块。 Error loading "D:\\anaconda3\\lib\\site-packages\\torch\\lib\\caffe2_nvrtc.dll" or one of its dependencies.

Python Console

> python
Python 3.9.13 (main, Aug 25 2022, 23:51:50) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch as th
>>> th.cuda.is_available()
True
>>> x = th.rand(5, 3).cuda()
>>> x
tensor([[0.3741, 0.8242, 0.1863],
        [0.6231, 0.3881, 0.3531],
        [0.9701, 0.2548, 0.5144],
        [0.3849, 0.0291, 0.2780],
        [0.8660, 0.4759, 0.3139]], device='cuda:0')

然后突然想起来，昨天晚上更新了CUDA驱动，主要是NVIDIA GeForce Experience推荐我更新的，没在意就直接点了。

通过nvidia-smi看一下CUDA版本

> nvidia-smi
Thu Feb 23 07:57:34 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 528.49       Driver Version: 528.49       CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ... WDDM  | 00000000:01:00.0  On |                  N/A |
| 30%   22C    P8    21W / 350W |    819MiB / 24576MiB |      2%      Default |
|                               |                      |                  N/A |
|    0   N/A  N/A     19972    C+G   ...artMenuExperienceHost.exe    N/A      |
|    0   N/A  N/A     22428    C+G   ...kyb3d8bbwe\\HxAccounts.exe    N/A      |
|    0   N/A  N/A     25204    C+G   ...ekyb3d8bbwe\\HxOutlook.exe    N/A      |
|    0   N/A  N/A     25456    C+G   D:\\Paoluz\\Paoluz.exe            N/A      |
|    0   N/A  N/A     25528    C+G   ...d\\runtime\\WeChatAppEx.exe    N/A      |
|    0   N/A  N/A     25864    C+G   ...e\\PhoneExperienceHost.exe    N/A      |
|    0   N/A  N/A     26960    C+G   ...8wekyb3d8bbwe\\Cortana.exe    N/A      |
|    0   N/A  N/A     28324    C+G   ...me\\Application\\chrome.exe    N/A      |
+-----------------------------------------------------------------------------+

果然CUDA Version变成了12.0，我记得之前是11.7，所以降级就可以了。（我也想升级，但是当前时间点PyTorch暂时还不支持12.0）

下载CUDA Toolkit 11.7

更新完之后重启电脑就可以了，神奇。