Tensorflow 无法打开 libcuda.so.1

Posted 2023-04-15

【中文标题】Tensorflow 无法打开 libcuda.so.1

【英文标题】：Tensorflow cannot open libcuda.so.1

【发布时间】：2017-06-12 22:23:28

【问题描述】：

我有一台配备 GeForce 940 MX 的笔记本电脑。我想在 GPU 上启动并运行 Tensorflow。我从他们的教程页面安装了所有东西，现在当我导入 Tensorflow 时，我得到了

>>> import tensorflow as tf
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened  CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:119] Couldn't open CUDA library libcuda.so.1. LD_LIBRARY_PATH: 
I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:165] hostname: workLaptop
I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:189] libcuda reported version is: Not found: was unable to find libcuda.so DSO loaded into this program
I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:193] kernel reported version is: Permission denied: could not open driver version path for reading: /proc/driver/nvidia/version
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1092] LD_LIBRARY_PATH: 
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1093] failed to find libcuda.so on this system: Failed precondition: could not dlopen DSO: libcuda.so.1; dlerror: libnvidia-fatbinaryloader.so.367.57: cannot open shared object file: No such file or directory
 I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so locally
>>> 

之后我认为它只是切换到在 cpu 上运行。

编辑：在我对所有东西都进行了核对之后，从头开始。现在我明白了：

>>> import tensorflow
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:119] Couldn't open CUDA library libcuda.so.1. LD_LIBRARY_PATH: :/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64
I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:165] hostname: workLaptop
I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:189] libcuda reported version is: Not found: was unable to find libcuda.so DSO loaded into this program
I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:193] kernel reported version is: Permission denied: could not open driver version path for reading: /proc/driver/nvidia/version
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1092] LD_LIBRARY_PATH: :/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1093] failed to find libcuda.so on this system: Failed precondition: could not dlopen DSO: libcuda.so.1; dlerror: libnvidia-fatbinaryloader.so.367.57: cannot open shared object file: No such file or directory
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so locally

【问题讨论】：

你真的安装了NVIDIA驱动吗？ libcuda 是驱动程序的一部分，而不是 CUDA 工具包

使用find /usr/ -name 'libcuda.so.1' 定位文件是否在我假设您添加到LD_LIBRARY_PATH 的标准cuda 库目录中？如果没有，只需在 cuda lib 目录中创建一个指向它的符号链接。

/usr/lib/x86_64-linux-gnu/libcuda.so.1 和 /usr/lib/i386-linux-gnu/libcuda.so.1。 cuda lib 目录到底在哪里？

我再重复一遍``could not open driver version path for reading: /proc/driver/nvidia/version" 表示您在运行 Tensorflow 时没有功能正常的 CUDA 驱动程序

这看起来不像是 Tensorflow 问题；相反，您似乎没有正确安装和运行 NVidia 驱动程序。一项测试：尝试运行“nvidia-smi”。如果驱动程序安装正确，它应该打印可用 GPU 的列表。

【参考方案1】：

libcuda.so.1 是指向特定于 NVIDIA 驱动程序版本的文件的符号链接。它可能指向错误的版本或它可能不存在。

# See where the link is pointing.  
ls  /usr/lib/x86_64-linux-gnu/libcuda.so.1 -la
# My result:
# lrwxrwxrwx 1 root root 19 Feb 22 20:40 \
# /usr/lib/x86_64-linux-gnu/libcuda.so.1 -> ./libcuda.so.375.39

# Make sure it is pointing to the right version. 
# Compare it with the installed NVIDIA driver.
nvidia-smi

# Replace libcuda.so.1 with a link to the correct version
cd /usr/lib/x86_64-linux-gnu
sudo ln -f -s libcuda.so.<yournvidia.version> libcuda.so.1

现在以同样的方式，从 libcuda.so.1 创建另一个符号链接到 LD_LIBRARY_PATH directory 中的同名链接。

您可能还会发现需要在 /usr/lib/x86_64-linux-gnu 中创建一个指向 libcuda.so.1 的链接，名为 libcuda.so

【讨论】：

现在以同样的方式，从 libcuda.so.1 创建另一个符号链接到 LD_LIBRARY_PATH 目录中的同名链接。 如何究竟是这样做的吗？我的“LD_LIBRARY_PATH 目录”是什么？非常感谢！

【参考方案2】：

万一还有人遇到这种情况。首先确保添加 --runtime=nvidia 参数以运行您的容器。

docker run --runtime=nvidia -t tensorflow/serving:latest-gpu

tensorflow/serving:latest-gpu 是 docker 镜像的名称。

【参考方案3】：

在我刚刚解决的情况下，它是将 GPU 驱动程序更新到最新版本并安装 cuda 工具包。首先，添加 ppa 并安装 GPU 驱动：

sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt update
sudo apt install nvidia-390

添加 ppa 后，它显示了驱动程序版本的选项，并且 390 是显示的最新“稳定”版本。

然后安装cuda工具包：

sudo apt install nvidia-cuda-toolkit

然后重启：

sudo reboot

它将驱动程序更新为比第一步中最初安装的 390 更新的版本（它是 410；这是 AWS 上的 p2.xlarge 实例）。