图像分类器ValueError：检查目标时出错：预期dense_31有2维，但得到的数组形状为（1463、224、224、3）

Posted 2023-03-12

【中文标题】图像分类器ValueError：检查目标时出错：预期dense_31有2维，但得到的数组形状为（1463、224、224、3）

【英文标题】：Image classifier ValueError: Error when checking target: expected dense_31 to have 2 dimensions, but got array with shape (1463, 224, 224, 3)

【发布时间】：2020-05-12 05:57:54

【问题描述】：

我正在尝试从多级数据的数据集中预测电影的类型。输入是这样的

Id              Genre             Action    Adventure   Animation   Biography   Comedy  Crime   Documentary  Drama  Family  Fantasy History Horror  Music       
tt0086425   ['Comedy', 'Drama']    0           0           0           0           1      0         0          1       0      0       0        0      0

like this 25 columns of genre are there for each movie poster

我已经完成了 EDA，现在尝试为这个多级数据制作预测模型，我的训练和测试分离看起来像这样

y = np.array(movies.drop(['Id', 'Genre'],axis=1))

from skmultilearn.model_selection import iterative_train_test_split
X_train, X_test, y_train, y_test = iterative_train_test_split(X, y, test_size=0.2)

X_train.shape, X_test.shape, y_train.shape, y_test.shape

o/p : ((5791, 224, 224, 3), (5791, 25), (1463, 224, 224, 3), (1463, 25))

y_test[0]

o/p :array([0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0], dtype=int64)

pd.DataFrame(
    'train': Counter(str(combination) for row in get_combination_wise_output_matrix(X_test, order=2) for combination in row),
    'test' : Counter(str(combination) for row in get_combination_wise_output_matrix(y_test, order=2) for combination in row)
).T.fillna(0.0)

o/p:

    (0, 0)  (0, 1)  (0, 10) (0, 11) (0, 12) (0, 13) (0, 14) (0, 18) (0, 19) (0, 2)  ... (9, 13) (9, 14) (9, 18) (9, 19) (9, 20) (9, 21) (9, 22) (9, 23) (9, 24) (9, 9)
train   1074.0  323.0   11.0    56.0    2.0 4.0 41.0    40.0    148.0   33.0    ... 4.0 18.0    32.0    7.0 3.0 1.0 14.0    1.0 0.0 370.0
test    269.0   81.0    2.0 14.0    1.0 0.0 7.0 10.0    37.0    4.0 ... 2.0 4.0 20.0    2.0 0.0 0.0 4.0 0.0 1.0 97.0

2 rows × 228 columns

现在为了创建预测图像分类器，我编写了以下代码：

model = Sequential()
model.add(Conv2D(filters=16, kernel_size=(5, 5), activation="relu", input_shape=(224,224,3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(filters=32, kernel_size=(5, 5), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(filters=64, kernel_size=(5, 5), activation="relu"))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(filters=64, kernel_size=(5, 5), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(25, activation='sigmoid'))

model.summary()

Model: "sequential_12"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_41 (Conv2D)           (None, 220, 220, 16)      1216      
_________________________________________________________________
max_pooling2d_41 (MaxPooling (None, 110, 110, 16)      0         
_________________________________________________________________
dropout_47 (Dropout)         (None, 110, 110, 16)      0         
_________________________________________________________________
conv2d_42 (Conv2D)           (None, 106, 106, 32)      12832     
_________________________________________________________________
max_pooling2d_42 (MaxPooling (None, 53, 53, 32)        0         
_________________________________________________________________
dropout_48 (Dropout)         (None, 53, 53, 32)        0         
_________________________________________________________________
conv2d_43 (Conv2D)           (None, 49, 49, 64)        51264     
_________________________________________________________________
max_pooling2d_43 (MaxPooling (None, 24, 24, 64)        0         
_________________________________________________________________
dropout_49 (Dropout)         (None, 24, 24, 64)        0         
_________________________________________________________________
conv2d_44 (Conv2D)           (None, 20, 20, 64)        102464    
_________________________________________________________________
max_pooling2d_44 (MaxPooling (None, 10, 10, 64)        0         
_________________________________________________________________
dropout_50 (Dropout)         (None, 10, 10, 64)        0         
_________________________________________________________________
flatten_12 (Flatten)         (None, 6400)              0         
_________________________________________________________________
dense_29 (Dense)             (None, 128)               819328    
_________________________________________________________________
dropout_51 (Dropout)         (None, 128)               0         
_________________________________________________________________
dense_30 (Dense)             (None, 64)                8256      
_________________________________________________________________
dropout_52 (Dropout)         (None, 64)                0         
_________________________________________________________________
dense_31 (Dense)             (None, 25)                1625      
=================================================================
Total params: 996,985
Trainable params: 996,985
Non-trainable params: 0
_________________________________________________________________

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

但是一旦我运行模型拟合，我就会得到值错误：

model.fit(X_train, y_train, epochs=10, validation_data=(X_test, y_test), batch_size=64)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\keras\engine\training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_array_lengths, batch_size)
    793                 feed_output_shapes,
    794                 check_batch_axis=False,  # Don't enforce the batch size.
--> 795                 exception_prefix='target')
    796 
    797             # Generate sample-wise weight values given the `sample_weight` and

~\AppData\Local\Continuum\anaconda3\lib\site-packages\keras\engine\training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
    129                         ': expected ' + names[i] + ' to have ' +
    130                         str(len(shape)) + ' dimensions, but got array '
--> 131                         'with shape ' + str(data_shape))
    132                 if not check_batch_axis:
    133                     data_shape = data_shape[1:]

ValueError: Error when checking target: expected dense_31 to have 2 dimensions, but got array with shape (1463, 224, 224, 3)

我无法弄清楚最后一步如何解决这个问题以及我在哪里犯了错误，好像已经在 conv2d 和密集层之间添加了 flatten 层，但是它没有修复。任何帮助将不胜感激

【参考方案1】：

为上述解决方案做出贡献，您还有一个错误：

model.add(Dense(25, activation='sigmoid'))

由于您有一个多类分类问题，正确的激活函数是softmax，而不是sigmoid。

因此，将您的行更改为

model.add(Dense(25, activation='softmax')).

【讨论】：

是的，这里必须考虑“softmax”，谢谢……我也运行了我的模型，准确率约为 91%，但是看看我的代码，根据你的经验，你能给出你的是否可以通过更改某些架构来提高其准确性？

如果数据集不平衡，准确度指标可能会产生误导；在这种情况下，F1-Score 等指标可能更相关。如果数据集非常平衡，那么 91% 的准确率是非常好的。原则上，您可以通过更改输入数据或更改架构来提高分数。如果我是你，我会从迁移学习开始，并使用成熟的架构，例如 DenseNet/ResNet。

谢谢我会尝试 twigging 架构以及使用 DenseNet..还必须用 val 准确度来绘制我的火车准确度...如果可能有过度拟合和高精度会误导

【参考方案2】：

你的错误在于你的目标：

X_train.shape, X_test.shape, y_train.shape, y_test.shape

o/p : ((5791, 224, 224, 3), (5791, 25), (1463, 224, 224, 3), (1463, 25))

您的 y_train 形状是 (1463, 224, 224, 3) 而不是 (5791, 25)，您的顺序有误。

然后解决方法是用以下方式替换分割线：

X_train, y_train, X_test, y_test = iterative_train_test_split(X, y, test_size=0.2)

Here is the doc