Cupy 错误 - 超出磁盘配额 [包括最小示例]

Posted 2023-02-16

【中文标题】Cupy 错误 - 超出磁盘配额 [包括最小示例]

【英文标题】：Cupy error - disk quota exceeded [Minimal example included]

【发布时间】：2019-08-01 07:36:16

【问题描述】：

在 cupy 中执行以下最小示例。

import cupy, cupyx

x = cupy.array([1., 2., 3.])
y = cupy.arange(10)
print(cupyx.get_runtime_info())

以下方式

(venv) user@ailx216:/work/sandbox$ CUDA_PATH=/usr/local/cuda-9.1 python cupy_test.py

给我一​​个奇怪的磁盘配额超出错误。

Traceback (most recent call last):
  File "cupy_test.py", line 4, in <module>
    y = cupy.arange(10)
  File "/home/sysdata/USR/anaconda3/envs/bleedingEdge/lib/python3.7/site-packages/cupy/creation/ranges.py", line 57, in arange
    _arange_ufunc(typ(start), typ(step), ret, dtype=dtype)
  File "cupy/core/_kernel.pyx", line 851, in cupy.core._kernel.ufunc.__call__
  File "cupy/core/_kernel.pyx", line 863, in cupy.core._kernel.ufunc._get_ufunc_kernel
  File "cupy/core/_kernel.pyx", line 625, in cupy.core._kernel._get_ufunc_kernel
  File "cupy/core/_kernel.pyx", line 60, in cupy.core._kernel._get_simple_elementwise_kernel
  File "cupy/core/carray.pxi", line 166, in cupy.core.core.compile_with_cache
  File "/home/sysdata/USR/anaconda3/envs/bleedingEdge/lib/python3.7/site-packages/cupy/cuda/compiler.py", line 177, in compile_with_cache
    temp_path = tf.name
  File "/home/sysdata/USR/anaconda3/envs/bleedingEdge/lib/python3.7/tempfile.py", line 499, in __exit__
    result = self.file.__exit__(exc, value, tb)
OSError: [Errno 122] Disk quota exceeded

我的设置是

> pip freeze
certifi==2019.6.16
chainer==5.3.0
cupy-cuda91==6.2.0
cycler==0.10.0
fastrlock==0.4
filelock==3.0.12
kiwisolver==1.1.0
matplotlib==3.1.1
numpy==1.16.4
olefile==0.46
pandas==0.25.0
patsy==0.5.1
Pillow==6.1.0
protobuf==3.9.0
pudb==2019.1
Pygments==2.4.2
pyparsing==2.4.2
python-dateutil==2.8.0
pytz==2019.1
scipy==1.3.0
seaborn==0.9.0
six==1.12.0
statsmodels==0.10.1
tornado==6.0.3
tqdm==4.32.2
urwid==2.0.1

我的 anaconda 安装在 A./home/sysdata/USR/anaconda3 我的工作目录是 B. /work/sandbox

两者都有足够的磁盘配额。我该如何调试呢？ 编辑：通过清除临时目录解决此错误。

但是现在我得到了错误：

  File "chainer_test.py", line 14, in <module>
    mydata_like = xp.zeros_like(mydata)
/cupy/creation/basic.py", line 205, in zeros_like
    order, strides, memptr = _new_like_order_and_strides(a, dtype, order)
/cupy/creation/basic.py", line 35, in _new_like_order_and_strides
    order = chr(_update_order_char(a, ord(order)))
TypeError: Argument 'x' has incorrect type (expected cupy.core.core.ndarray, got numpy.ndarray)

print(cupyx.get_runtime_info())的输出如下：

CuPy Version          : 6.2.0
CUDA Root             : /usr/local/cuda-9.1/
CUDA Build Version    : 9010
CUDA Driver Version   : 10000
CUDA Runtime Version  : 9010
cuDNN Build Version   : 7102
cuDNN Version         : 7102
NCCL Build Version    : 2115
NCCL Runtime Version  : (unknown)

【参考方案1】：

即使有足够的磁盘空间，也可能发生“超出磁盘配额”。

一种可能是 inode 限制：https://superuser.com/questions/959175/disk-quota-exceeded-when-writing-to-tmp-but-plenty-of-space-linux

如果清除 cupy 内核缓存（~/.cupy/kernel_cache）会怎样？