解决Check failed: cudnnSetTensorNdDescriptor(handle_.get(), elem_type, nd, dims.data(), strides.data()

Posted 2022-01-22 沉迷单车的追风少年

项目场景：

tensorflow 加载预训练模型 InceptionV3， 计算FID

---

问题描述：

2021-12-13 07:44:25.012505: F tensorflow/stream_executor/cuda/cuda_dnn.cc:533] Check failed: cudnnSetTensorNdDescriptor(handle_.get(), elem_type, nd, dims.data(), strides.data()) == CUDNN_STATUS_SUCCESS (3 vs. 0)batch_descriptor: count: 96 feature_map_count: 288 spatial: 0 0  value_min: 0.000000 value_max: 0.000000 layout: BatchYXDepth
Aborted (core dumped)

问题代码：

import numpy as np
import tensorflow as tf
from scipy.linalg import sqrtm

BATCH_SIZE = 64
inception_model = tf.keras.applications.InceptionV3(include_top=False,
                                                    weights="imagenet",
                                                    pooling='avg')
# inception_model = tf.keras.applications.inception_v3.InceptionV3(
#     include_top=False,
#     weights='imagenet',
#     pooling='avg'
# )

x = np.random.randint(low=0, high=255, size=(96, 32, 32, 3)).astype('float32')
result = inception_model(x)
print(f"result shape result.shape")
exit()

---

原因分析：

输入的input太小，做卷积运算的时候input会越来越小，过于小会“无法卷”，报错。

---

解决方案：

将input变大即可。

import numpy as np
import tensorflow as tf
from scipy.linalg import sqrtm

BATCH_SIZE = 64
inception_model = tf.keras.applications.InceptionV3(include_top=False,
                                                    weights="imagenet",
                                                    pooling='avg')
# inception_model = tf.keras.applications.inception_v3.InceptionV3(
#     include_top=False,
#     weights='imagenet',
#     pooling='avg'
# )

x = np.random.randint(low=0, high=255, size=(96, 640, 480, 3)).astype('float32')
result = inception_model(x)
print(f"result shape result.shape")
exit()