MaxPooling2D、Conv2D、UpSampling2D层的输出大小是如何计算的？

Posted 2023-02-16

【中文标题】MaxPooling2D、Conv2D、UpSampling2D层的输出大小是如何计算的？

【英文标题】：How are the output size of MaxPooling2D, Conv2D, UpSampling2D layers calculated?

【发布时间】：2019-06-22 16:48:26

【问题描述】：

我正在学习卷积自动编码器，并且正在使用 keras 构建图像降噪器。 以下代码可用于构建模型：

denoiser.add(Conv2D(32, (3,3), input_shape=(28,28,1), padding='same')) 
denoiser.add(Activation('relu'))
denoiser.add(MaxPooling2D(pool_size=(2,2)))

denoiser.add(Conv2D(16, (3,3), padding='same'))
denoiser.add(Activation('relu'))
denoiser.add(MaxPooling2D(pool_size=(2,2)))

denoiser.add(Conv2D(8, (3,3), padding='same'))
denoiser.add(Activation('relu'))

################## HEY WHAT NO MAXPOOLING?

denoiser.add(Conv2D(8, (3,3), padding='same'))
denoiser.add(Activation('relu'))
denoiser.add(UpSampling2D((2,2)))

denoiser.add(Conv2D(16, (3,3), padding='same'))
denoiser.add(Activation('relu'))
denoiser.add(UpSampling2D((2,2)))

denoiser.add(Conv2D(1, (3,3), padding='same'))

denoiser.compile(optimizer='adam', loss='mean_squared_error', metrics=['accuracy'])
denoiser.summary()

并给出如下总结：

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_155 (Conv2D)          (None, 28, 28, 32)        320       
_________________________________________________________________
activation_162 (Activation)  (None, 28, 28, 32)        0         
_________________________________________________________________
max_pooling2d_99 (MaxPooling (None, 14, 14, 32)        0         
_________________________________________________________________
conv2d_156 (Conv2D)          (None, 14, 14, 16)        4624      
_________________________________________________________________
activation_163 (Activation)  (None, 14, 14, 16)        0         
_________________________________________________________________
max_pooling2d_100 (MaxPoolin (None, 7, 7, 16)          0         
_________________________________________________________________
conv2d_157 (Conv2D)          (None, 7, 7, 8)           1160      
_________________________________________________________________
activation_164 (Activation)  (None, 7, 7, 8)           0         
_________________________________________________________________
conv2d_158 (Conv2D)          (None, 7, 7, 8)           584       
_________________________________________________________________
activation_165 (Activation)  (None, 7, 7, 8)           0         
_________________________________________________________________
up_sampling2d_25 (UpSampling (None, 14, 14, 8)         0         
_________________________________________________________________
conv2d_159 (Conv2D)          (None, 14, 14, 16)        1168      
_________________________________________________________________
activation_166 (Activation)  (None, 14, 14, 16)        0         
_________________________________________________________________
up_sampling2d_26 (UpSampling (None, 28, 28, 16)        0         
_________________________________________________________________
conv2d_160 (Conv2D)          (None, 28, 28, 1)         145       
=================================================================
Total params: 8,001
Trainable params: 8,001
Non-trainable params: 0
_________________________________________________________________

我不确定如何计算 MaxPooling2D、Conv2D、UpSampling2D 输出大小。我已经阅读了 keras 文档，但我仍然感到困惑。有很多参数会影响输出形状，比如 Conv2D 层的stride 或padding，我不知道它究竟是如何影响输出形状的。

我不明白为什么在注释行之前没有MaxPooling2D 层。编辑代码以在注释上方包含 convmodel3.add(MaxPooling2D(pool_size=(2,2))) 层，它将最终输出形状变为 (None, 12, 12, 1)

编辑代码以在注释前包含 convmodel3.add(MaxPooling2D(pool_size=(2,2))) 层，然后 convmodel3.add(UpSampling2D((2,2))) 将最终输出变为 (None, 24, 24, 1)。这不应该是 (None, 28, 28, 1) 吗？ 代码和总结：

convmodel3 = Sequential()
convmodel3.add(Conv2D(32, (3,3), input_shape=(28,28,1), padding='same')) 
convmodel3.add(Activation('relu'))
convmodel3.add(MaxPooling2D(pool_size=(2,2)))

convmodel3.add(Conv2D(16, (3,3), padding='same'))
convmodel3.add(Activation('relu'))
convmodel3.add(MaxPooling2D(pool_size=(2,2)))

convmodel3.add(Conv2D(8, (3,3), padding='same'))
convmodel3.add(Activation('relu'))
convmodel3.add(MaxPooling2D(pool_size=(2,2))) # ADDED MAXPOOL

################## HEY WHAT NO MAXPOOLING?

convmodel3.add(UpSampling2D((2,2))) # ADDED UPSAMPLING
convmodel3.add(Conv2D(16, (3,3), padding='same'))
convmodel3.add(Activation('relu'))
convmodel3.add(UpSampling2D((2,2)))

convmodel3.add(Conv2D(32, (3,3), padding='same'))
convmodel3.add(Activation('relu'))
convmodel3.add(UpSampling2D((2,2)))

convmodel3.add(Conv2D(1, (3,3), padding='same'))

convmodel3.compile(optimizer='adam', loss='mean_squared_error', metrics=['accuracy'])
convmodel3.summary()

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_247 (Conv2D)          (None, 28, 28, 32)        320       
_________________________________________________________________
activation_238 (Activation)  (None, 28, 28, 32)        0         
_________________________________________________________________
max_pooling2d_141 (MaxPoolin (None, 14, 14, 32)        0         
_________________________________________________________________
conv2d_248 (Conv2D)          (None, 14, 14, 16)        4624      
_________________________________________________________________
activation_239 (Activation)  (None, 14, 14, 16)        0         
_________________________________________________________________
max_pooling2d_142 (MaxPoolin (None, 7, 7, 16)          0         
_________________________________________________________________
conv2d_249 (Conv2D)          (None, 7, 7, 8)           1160      
_________________________________________________________________
activation_240 (Activation)  (None, 7, 7, 8)           0         
_________________________________________________________________
max_pooling2d_143 (MaxPoolin (None, 3, 3, 8)           0         
_________________________________________________________________
up_sampling2d_60 (UpSampling (None, 6, 6, 8)           0         
_________________________________________________________________
conv2d_250 (Conv2D)          (None, 6, 6, 16)          1168      
_________________________________________________________________
activation_241 (Activation)  (None, 6, 6, 16)          0         
_________________________________________________________________
up_sampling2d_61 (UpSampling (None, 12, 12, 16)        0         
_________________________________________________________________
conv2d_251 (Conv2D)          (None, 12, 12, 32)        4640      
_________________________________________________________________
activation_242 (Activation)  (None, 12, 12, 32)        0         
_________________________________________________________________
up_sampling2d_62 (UpSampling (None, 24, 24, 32)        0         
_________________________________________________________________
conv2d_252 (Conv2D)          (None, 24, 24, 1)         289       
=================================================================
Total params: 12,201
Trainable params: 12,201
Non-trainable params: 0
_________________________________________________________________

None 在输出形状中有什么意义？

另外，编辑Conv2D 层以不包括填充，会引发错误：

ValueError: Negative dimension size caused by subtracting 3 from 2 for 'conv2d_240/convolution' (op: 'Conv2D') with input shapes: [?,2,2,16], [3,3,16,32].

为什么？

【问题讨论】：

您可能会发现这篇论文很有用：arxiv.org/abs/1603.07285，与此 repo 中的插图一起使用：github.com/vdumoulin/conv_arithmetic

【参考方案1】：

对于卷积（此处为 2D）层，要考虑的重点是图像的体积（宽度 x 高度 x 深度）和您给它的四个参数。这些参数是

过滤器数量 K 过滤器大小（空间）F 过滤器在 S 处移动的步幅 零填充P

输出形状的公式为

Wnew = (W - F + 2*P)/S + 1 新 = (H - F + 2*P)/S + 1 Dnew = K

这取自这个线程what is the effect of tf.nn.conv2d() on an input tensor shape?，关于零填充等的更多信息可以在那里找到。

对于最大池化和上采样，大小仅受池大小和步幅的影响。在您的示例中，您的池大小为 (2,2) 并且未定义步幅（因此默认为池大小，请参见此处https://keras.io/layers/pooling/）。上采样的工作原理相同。池大小只需要一个 2x2 像素的池，找到它们的总和并将它们放入一个像素中。因此将 2x2 像素转换为 1x1 像素，对其进行编码。上采样是同样的事情，但不是对像素值求和，而是在池中重复这些值。

您没有最大池化层以及图像尺寸在您的情况下混乱的原因是由于该阶段的图像大小。查看网络，图像尺寸已经是 [7,7,8]。池大小和步幅分别为 (2,2) 和 2，这会将图像的分辨率降低到 [3,3,8]。在上采样层之后，维度将从 3 -> 6 -> 12 -> 24 变为每行和每列丢失 4 个像素。

None 的重要性（如果我错了，请纠正我，我不是 100% 确定）是由于网络通常在卷积层期望多个图像。通常预期的维度是

[Number of images, Width, Height, Depth]

因此，第一个元素被指定为 none 的原因是您的网络一次只期望一个图像，因此它被指定为 None（我再次不确定这一点）。