Keras ValueError：尺寸必须相等问题

Posted 2023-03-12

【中文标题】Keras ValueError：尺寸必须相等问题

【英文标题】：Keras ValueError: Dimensions must be equal issue

【发布时间】：2019-10-11 14:57:27

【问题描述】：

即使在 answer 和 cmets 中应用了建议后，尺寸不匹配问题似乎仍然存在。这也是要复制的确切代码和数据文件：https://drive.google.com/drive/folders/1q67s0VhB-O7J8OtIhU2jmj7Kc4LxL3sf?usp=sharing

如何解决这个问题！？最新代码，模型摘要，使用的功能和我得到的错误如下

type_ae=='dcor'
#Wrappers for keras
def custom_loss1(y_true,y_pred):
    dcor = -1*distance_correlation(y_true,encoded_layer)
    return dcor

def custom_loss2(y_true,y_pred):
    recon_loss = losses.categorical_crossentropy(y_true, y_pred)
    return recon_loss

input_layer =  Input(shape=(64,64,1))

encoded_layer = Conv2D(filters = 128, kernel_size = (5,5),padding = 'same',activation ='relu', 
                       input_shape = (64,64,1))(input_layer)
encoded_layer = MaxPool2D(pool_size=(2,2))(encoded_layer)
encoded_layer = Dropout(0.25)(encoded_layer)
encoded_layer = (Conv2D(filters = 64, kernel_size = (3,3),padding = 'same',activation ='relu'))(encoded_layer)
encoded_layer = (MaxPool2D(pool_size=(2,2)))(encoded_layer)
encoded_layer = (Dropout(0.25))(encoded_layer)

encoded_layer = (Conv2D(filters = 64, kernel_size = (3,3),padding = 'same',activation ='relu'))(encoded_layer)
encoded_layer = (MaxPool2D(pool_size=(2,2)))(encoded_layer)
encoded_layer = (Dropout(0.25))(encoded_layer)
encoded_layer = Conv2D(filters = 1, kernel_size = (3,3),padding = 'same',activation ='relu', 
                       input_shape = (64,64,1),strides=1)(encoded_layer)
encoded_layer = ZeroPadding2D(padding=(28, 28), data_format=None)(encoded_layer)

decoded_imag = Conv2D(8, (2, 2), activation='relu', padding='same')(encoded_layer)
decoded_imag = UpSampling2D((2, 2))(decoded_imag)
decoded_imag = Conv2D(8, (3, 3), activation='relu', padding='same')(decoded_imag)
decoded_imag = UpSampling2D((2, 2))(decoded_imag)
decoded_imag = Conv2D(16, (3, 3), activation='relu', padding='same')(decoded_imag)
decoded_imag = UpSampling2D((2, 2))(decoded_imag)
decoded_imag = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(decoded_imag)
flat_layer = Flatten()(decoded_imag)
dense_layer = Dense(256,activation = "relu")(flat_layer)          
dense_layer = Dense(64,activation = "relu")(dense_layer) 
dense_layer = Dense(32,activation = "relu")(dense_layer) 
output_layer = Dense(9, activation = "softmax")(dense_layer)
autoencoder = Model(input_layer, [encoded_layer,output_layer])
autoencoder.summary()
autoencoder.compile(optimizer='adadelta', loss=[custom_loss1,custom_loss2])
autoencoder.fit(x_train,[x_train, y_train],batch_size=32,epochs=3,shuffle=True,
                validation_data=(x_val, [x_val,y_val]))

数据的维度：

x_train.shape:  (4000, 64, 64, 1)
x_val.shape:  (1000, 64, 64, 1)
y_train.shape:  (4000, 9)
y_val.shape:  (1000, 9)

损失看起来像：

def custom_loss1(y_true,y_pred):
    dcor = -1*distance_correlation(y_true,encoded_layer)
    return dcor

def custom_loss2(y_true,y_pred):
    recon_loss = losses.categorical_crossentropy(y_true, y_pred)
    return recon_loss

基于张量的相关函数如下：

def distance_correlation(y_true,y_pred):
    pred_r = tf.reduce_sum(y_pred*y_pred,1)
    pred_r = tf.reshape(pred_r,[-1,1])
    pred_d = pred_r - 2*tf.matmul(y_pred,tf.transpose(y_pred))+tf.transpose(pred_r)
    true_r = tf.reduce_sum(y_true*y_true,1)
    true_r = tf.reshape(true_r,[-1,1])
    true_d = true_r - 2*tf.matmul(y_true,tf.transpose(y_true))+tf.transpose(true_r)
    concord = 1-tf.matmul(y_true,tf.transpose(y_true))
    #print(pred_d)
    #print(tf.reshape(tf.reduce_mean(pred_d,1),[-1,1]))
    #print(tf.reshape(tf.reduce_mean(pred_d,0),[1,-1]))
    #print(tf.reduce_mean(pred_d))
    tf.check_numerics(pred_d,'pred_d has NaN')
    tf.check_numerics(true_d,'true_d has NaN')
    A = pred_d - tf.reshape(tf.reduce_mean(pred_d,1),[-1,1]) - tf.reshape(tf.reduce_mean(pred_d,0),[1,-1]) + tf.reduce_mean(pred_d)
    B = true_d - tf.reshape(tf.reduce_mean(true_d,1),[-1,1]) - tf.reshape(tf.reduce_mean(true_d,0),[1,-1]) + tf.reduce_mean(true_d)
    #dcor = -tf.reduce_sum(concord*pred_d)/tf.reduce_sum((1-concord)*pred_d)
    dcor = -tf.log(tf.reduce_mean(A*B))+tf.log(tf.sqrt(tf.reduce_mean(A*A)*tf.reduce_mean(B*B)))#-tf.reduce_sum(concord*pred_d)/tf.reduce_sum((1-concord)*pred_d)
    #print(dcor.shape)
    #tf.Print(dcor,[dcor])
    #dcor = tf.tile([dcor],batch_size)
    return (dcor)

模型摘要如下：

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_5 (InputLayer)         (None, 64, 64, 1)         0         
_________________________________________________________________
conv2d_30 (Conv2D)           (None, 64, 64, 128)       3328      
_________________________________________________________________
max_pooling2d_13 (MaxPooling (None, 32, 32, 128)       0         
_________________________________________________________________
dropout_13 (Dropout)         (None, 32, 32, 128)       0         
_________________________________________________________________
conv2d_31 (Conv2D)           (None, 32, 32, 64)        73792     
_________________________________________________________________
max_pooling2d_14 (MaxPooling (None, 16, 16, 64)        0         
_________________________________________________________________
dropout_14 (Dropout)         (None, 16, 16, 64)        0         
_________________________________________________________________
conv2d_32 (Conv2D)           (None, 16, 16, 64)        36928     
_________________________________________________________________
max_pooling2d_15 (MaxPooling (None, 8, 8, 64)          0         
_________________________________________________________________
dropout_15 (Dropout)         (None, 8, 8, 64)          0         
_________________________________________________________________
conv2d_33 (Conv2D)           (None, 8, 8, 1)           577       
_________________________________________________________________
zero_padding2d_5 (ZeroPaddin (None, 64, 64, 1)         0         
_________________________________________________________________
conv2d_34 (Conv2D)           (None, 64, 64, 8)         40        
_________________________________________________________________
up_sampling2d_10 (UpSampling (None, 128, 128, 8)       0         
_________________________________________________________________
conv2d_35 (Conv2D)           (None, 128, 128, 8)       584       
_________________________________________________________________
up_sampling2d_11 (UpSampling (None, 256, 256, 8)       0         
_________________________________________________________________
conv2d_36 (Conv2D)           (None, 256, 256, 16)      1168      
_________________________________________________________________
up_sampling2d_12 (UpSampling (None, 512, 512, 16)      0         
_________________________________________________________________
conv2d_37 (Conv2D)           (None, 512, 512, 1)       145       
_________________________________________________________________
flatten_4 (Flatten)          (None, 262144)            0         
_________________________________________________________________
dense_13 (Dense)             (None, 256)               67109120  
_________________________________________________________________
dense_14 (Dense)             (None, 64)                16448     
_________________________________________________________________
dense_15 (Dense)             (None, 32)                2080      
_________________________________________________________________
dense_16 (Dense)             (None, 9)                 297       
=================================================================
Total params: 67,244,507
Trainable params: 67,244,507
Non-trainable params: 0
_________________________________________________________________

这是错误：

InvalidArgumentError                      Traceback (most recent call last)
~/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in _create_c_op(graph, node_def, inputs, control_inputs)
   1658   try:
-> 1659     c_op = c_api.TF_FinishOperation(op_desc)
   1660   except errors.InvalidArgumentError as e:

InvalidArgumentError: Dimensions must be equal, but are 1 and 64 for 'loss_1/zero_padding2d_5_loss/MatMul' (op: 'BatchMatMul') with input shapes: [?,64,64,1], [1,64,64,?].

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-11-0e924885fc6b> in <module>
     40 autoencoder = Model(input_layer, [encoded_layer,output_layer])
     41 autoencoder.summary()
---> 42 autoencoder.compile(optimizer='adadelta', loss=[custom_loss1,custom_loss2])
     43 autoencoder.fit(x_train,[x_train, y_train],batch_size=32,epochs=3,shuffle=True,
     44                 validation_data=(x_val, [x_val,y_val]))

~/anaconda3/lib/python3.6/site-packages/keras/engine/training.py in compile(self, optimizer, loss, metrics, loss_weights, sample_weight_mode, weighted_metrics, target_tensors, **kwargs)
    340                 with K.name_scope(self.output_names[i] + '_loss'):
    341                     output_loss = weighted_loss(y_true, y_pred,
--> 342                                                 sample_weight, mask)
    343                 if len(self.outputs) > 1:
    344                     self.metrics_tensors.append(output_loss)

~/anaconda3/lib/python3.6/site-packages/keras/engine/training_utils.py in weighted(y_true, y_pred, weights, mask)
    402         """
    403         # score_array has ndim >= 2
--> 404         score_array = fn(y_true, y_pred)
    405         if mask is not None:
    406             # Cast the mask to floatX to avoid float64 upcasting in Theano

<ipython-input-11-0e924885fc6b> in custom_loss1(y_true, y_pred)
      2 #Wrappers for keras
      3 def custom_loss1(y_true,y_pred):
----> 4         dcor = -1*distance_correlation(y_true,encoded_layer)
      5         return dcor
      6 

<ipython-input-6-f282528532cc> in distance_correlation(y_true, y_pred)
      2     pred_r = tf.reduce_sum(y_pred*y_pred,1)
      3     pred_r = tf.reshape(pred_r,[-1,1])
----> 4     pred_d = pred_r - 2*tf.matmul(y_pred,tf.transpose(y_pred))+tf.transpose(pred_r)
      5     true_r = tf.reduce_sum(y_true*y_true,1)
      6     true_r = tf.reshape(true_r,[-1,1])

~/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py in matmul(a, b, transpose_a, transpose_b, adjoint_a, adjoint_b, a_is_sparse, b_is_sparse, name)
   2415         adjoint_b = True
   2416       return gen_math_ops.batch_mat_mul(
-> 2417           a, b, adj_x=adjoint_a, adj_y=adjoint_b, name=name)
   2418 
   2419     # Neither matmul nor sparse_matmul support adjoint, so we conjugate

~/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py in batch_mat_mul(x, y, adj_x, adj_y, name)
   1421   adj_y = _execute.make_bool(adj_y, "adj_y")
   1422   _, _, _op = _op_def_lib._apply_op_helper(
-> 1423         "BatchMatMul", x=x, y=y, adj_x=adj_x, adj_y=adj_y, name=name)
   1424   _result = _op.outputs[:]
   1425   _inputs_flat = _op.inputs

~/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py in _apply_op_helper(self, op_type_name, name, **keywords)
    786         op = g.create_op(op_type_name, inputs, output_types, name=scope,
    787                          input_types=input_types, attrs=attr_protos,
--> 788                          op_def=op_def)
    789       return output_structure, op_def.is_stateful, op
    790 

~/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py in new_func(*args, **kwargs)
    505                 'in a future version' if date is None else ('after %s' % date),
    506                 instructions)
--> 507       return func(*args, **kwargs)
    508 
    509     doc = _add_deprecated_arg_notice_to_docstring(

~/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in create_op(***failed resolving arguments***)
   3298           input_types=input_types,
   3299           original_op=self._default_original_op,
-> 3300           op_def=op_def)
   3301       self._create_op_helper(ret, compute_device=compute_device)
   3302     return ret

~/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in __init__(self, node_def, g, inputs, output_types, control_inputs, input_types, original_op, op_def)
   1821           op_def, inputs, node_def.attr)
   1822       self._c_op = _create_c_op(self._graph, node_def, grouped_inputs,
-> 1823                                 control_input_ops)
   1824 
   1825     # Initialize self._outputs.

~/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in _create_c_op(graph, node_def, inputs, control_inputs)
   1660   except errors.InvalidArgumentError as e:
   1661     # Convert to ValueError for backwards compatibility.
-> 1662     raise ValueError(str(e))
   1663 
   1664   return c_op

ValueError: Dimensions must be equal, but are 1 and 64 for 'loss_1/zero_padding2d_5_loss/MatMul' (op: 'BatchMatMul') with input shapes: [?,64,64,1], [1,64,64,?].

【问题讨论】：

你能告诉我们custom_loss1和custom_loss2的代码

当然..将其添加到用于 custom_loss1 工作的导入（依赖）是“来自 dcor import distance_correlation”的位置

我想您已经知道错误的原因，正如我从下面的 cmets 中看到的那样。 custom_loss1 中的 encoded_layer 与 y_true 的形状不同。我看了你的模型，我对你到底想要达到什么感到困惑。也许如果你能解释一下，我可以建议你改变模型。

如果你想计算custom_loss1 wrt 原始图像，custom_loss2 wrt 你在fit 中传递的标量标签，我想我有一个解决方案给你。但我不确定你是否真的想这样做。

@Anakin 完全正确。我想要自定义损失 1 wrt 编码的原始图像。和自定义损失 2 wrt 标签......同时做适合

【参考方案1】：

您有两个损失函数，因此您必须通过两个y（基本事实）来评估与预测相关的损失。

您的第一个预测是层encoded_layer 的输出，其大小为(None, 8, 8, 128)，从conv2d_59 (Conv2D) 的model.summary 中观察到

但是您传递的适合y 的是[x_train, y_train]。 loss_1 期望输入大小为(None, 8, 8, 128)，但您传递的x_train 大小不同。

如果您希望loss_1 找到输入图像与编码图像的相关性，则堆叠卷积，使得卷积的输出将产生与您的 x_train 图像形状相同的形状。使用model.summary 查看卷积的输出形状。

不使用卷积层的填充、步幅和内核大小来获得所需的卷积输出大小。使用公式W2=(W1−F+2P)/S+1 和H2=(H1−F+2P)/S+1 找出卷积的输出宽度和高度。检查这个reference

---

您的方法存在两个主要问题。

您的损失函数正在检查编码图像和实际图像之间的相关性。正确的做法是从编码图像中解码回图像，然后检查解码图像与实际图像之间的相关性（以自动编码器的行数） 您的损失 1 正在使用 numpy 数组。要使损失函数成为计算图的一部分，它应该使用张量操作，而不是 numy 操作。

下面是工作代码。但是，对于损失 1，我使用两个图像的 l2 范数。如果你想使用相关性，那么你必须以某种方式将其转换为张量运算（这与这个问题不同）

def image_loss(y_true,y_pred):
    return tf.norm(y_true - y_pred)

def label_loss(y_true,y_pred):
    return categorical_crossentropy(y_true, y_pred)

input_img = Input(shape=(64, 64, 1))  

enocded_imag = Conv2D(16, (3, 3), activation='relu', padding='same')(input_img)
enocded_imag = MaxPooling2D((2, 2), padding='same')(enocded_imag)
enocded_imag = Conv2D(8, (3, 3), activation='relu', padding='same')(enocded_imag)
enocded_imag = MaxPooling2D((2, 2), padding='same')(enocded_imag)
enocded_imag = Conv2D(8, (3, 3), activation='relu', padding='same')(enocded_imag)
enocded_imag = MaxPooling2D((2, 2), padding='same')(enocded_imag)

decoded_imag = Conv2D(8, (2, 2), activation='relu', padding='same')(enocded_imag)
decoded_imag = UpSampling2D((2, 2))(decoded_imag)
decoded_imag = Conv2D(8, (3, 3), activation='relu', padding='same')(decoded_imag)
decoded_imag = UpSampling2D((2, 2))(decoded_imag)
decoded_imag = Conv2D(16, (3, 3), activation='relu', padding='same')(decoded_imag)
decoded_imag = UpSampling2D((2, 2))(decoded_imag)
decoded_imag = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(decoded_imag)

flat_layer = Flatten()(enocded_imag) 
dense_layer = Dense(32,activation = "relu")(flat_layer) 
output_layer = Dense(9, activation = "softmax")(dense_layer)

model = Model(input_img, [decoded_imag, output_layer])

model.compile(optimizer='adadelta', loss=[image_loss, label_loss])
images = np.random.randn(10,64,64,1)
model.fit(images, [images, np.random.randn(10,9)])

---

您编写的损失函数distance_correlation 假设y_true 和y_pred 中的每一行代表一个图像。当您使用Dense 层时，它会起作用，因为Dense 层会输出一批（行）向量，其中每个向量代表一个单独的图像。但是，2D 卷积在一批具有多个通道的 2d 张量上运行（您只有 1 个通道）。因此，要使用 distance_correlation 损失函数，您必须重塑张量，使每一行对应一个图像。添加以下两行以重塑您的张量。

def distance_correlation(y_true,y_pred):
    y_true = tf.reshape(tf.squeeze(y_true), [-1,64*64])
    y_pred = tf.reshape(tf.squeeze(y_pred), [-1,64*64])
    .... REST OF THE CODE ....

【讨论】：

堆叠卷积，这样卷积的输出将产生与您的 x_train 图像形状相同的形状：我使用 Reshape 吗？我对此有误。什么是确保该转换的结果与我的图像尺寸一致的好方法？确实，我正在尝试使用该卷积的图像和输出之间的这种相关性。我知道尺寸应该匹配。尽管要更改该 conv 层的输出尺寸，但我仍在苦苦挣扎。那怎么办？

更新了更多信息的答案

在您的回答中应用建议后我仍然有错误！根据我在应用您的答案后看到的内容更新了最后的问题。

将相关性 fn 更新为现在仅基于张量。再次在问题底部更新。仍然出现同样的错误！

@hearse 你能用你的“修改”代码更新模型构建代码吗？

【参考方案2】：

目的是使用custom_loss1 中的原始图像和custom_loss2 中的标量标签值。我认为@mujjiga 在他的回答中的工作代码几乎是正确的。我建议稍作修改。

在model.compile() 中传递需要它的损失中的输入张量。保持另一个相同。 model.fit() 只是传递标签。

model.compile(optimizer='adadelta', loss=[custom_loss1(input_layer), custom_loss2]) 
mode.fit(x_train, y_train)

自定义损失函数内部：

def custom_loss1(input):
    def loss1(y_true, y_pred):
        return tf.norm(input - y_pred) # use your custom loss 1
    return loss1

def custom_loss2(y_true, y_pred):
    return categorical_crossentropy(y_true, y_pred) # use your custom loss 2

首先尝试使用简单的内置 Keras 损失函数。如果效果不错，请查看您的自定义损失函数。

【讨论】：

images 将成为您的x_train

axis=1 追加列而不是追加行

主要区别是通过相应的损失函数只访问标签的切片，而不是全部使用。因此，在 loss1 中您访问原始图像，在 loss2 中您访问标量值。为了更好地理解，我已经用 cmets 编辑了我的答案。

在尝试您的建议时，特别是部分，np.append(x_train, y_train, axis =1) 给出了错误。我得到 ValueError: 所有输入数组必须具有相同的维数。 print(x_train.shape) 和 print(y_train.shape) 给出 (4000, 64, 64, 1) 和 (4000, 9) 作为形状。在 model.fit 中使用之前，必须正确完成附加。怎么修？这也是要复制的确切代码和数据文件：drive.google.com/drive/folders/…

好的，很抱歉之前的建议由于形状不匹配而不起作用。我在我的 train_data 和 label 都是数组的情况下使用它。我已经用另一种处理方式更新了答案。也许你可以试试这个。