比较 Conv2D 与 Tensorflow 和 PyTorch 之间的填充

Posted 2023-02-23

【中文标题】比较 Conv2D 与 Tensorflow 和 PyTorch 之间的填充

【英文标题】：Comparing Conv2D with padding between Tensorflow and PyTorch

【发布时间】：2019-03-29 06:33:27

【问题描述】：

我正在尝试将从 Tensorflow 模型中保存的权重导入 PyTorch。到目前为止，结果非常相似。当模型使用stride=2 调用conv2d 时，我遇到了障碍。

为了验证不匹配，我在 TF 和 PyTorch 之间进行了非常简单的比较。首先，我将conv2d 与stride=1 进行比较。

import tensorflow as tf
import numpy as np
import torch
import torch.nn.functional as F


np.random.seed(0)
sess = tf.Session()

# Create random weights and input
weights = torch.empty(3, 3, 3, 8)
torch.nn.init.constant_(weights, 5e-2)
x = np.random.randn(1, 3, 10, 10)

weights_tf = tf.convert_to_tensor(weights.numpy(), dtype=tf.float32)
# PyTorch adopts [outputC, inputC, kH, kW]
weights_torch = torch.Tensor(weights.permute((3, 2, 0, 1)))

# Tensorflow defaults to NHWC
x_tf = tf.convert_to_tensor(x.transpose((0, 2, 3, 1)), dtype=tf.float32)
x_torch = torch.Tensor(x)

# TF Conv2D
tf_conv2d = tf.nn.conv2d(x_tf,
                         weights_tf,
                         strides=[1, 1, 1, 1],
                         padding="SAME")

# PyTorch Conv2D
torch_conv2d = F.conv2d(x_torch, weights_torch, padding=1, stride=1)

sess.run(tf.global_variables_initializer())
tf_result = sess.run(tf_conv2d)

diff = np.mean(np.abs(tf_result.transpose((0, 3, 1, 2)) - torch_conv2d.detach().numpy()))
print('Mean of Abs Diff: 0'.format(diff))

本次执行的结果是：

Mean of Abs Diff: 2.0443112092038973e-08

当我将stride 更改为 2 时，结果开始变化。

# TF Conv2D
tf_conv2d = tf.nn.conv2d(x_tf,
                         weights_tf,
                         strides=[1, 2, 2, 1],
                         padding="SAME")

# PyTorch Conv2D
torch_conv2d = F.conv2d(x_torch, weights_torch, padding=1, stride=2)

本次执行的结果是：

Mean of Abs Diff: 0.2104552686214447

根据 PyTorch 文档，conv2d uses zero-padding 由 padding 参数定义。因此，在我的示例中，输入的左侧、顶部、右侧和底部都添加了零。

如果 PyTorch 只是根据输入参数在两侧添加填充，在 Tensorflow 中应该很容易复制。

# Manually add padding - consistent with PyTorch
paddings = tf.constant([[0, 0], [1, 1], [1, 1], [0, 0]])
x_tf = tf.convert_to_tensor(x.transpose((0, 2, 3, 1)), dtype=tf.float32)
x_tf = tf.pad(x_tf, paddings, "CONSTANT")

# TF Conv2D
tf_conv2d = tf.nn.conv2d(x_tf,
                         weights_tf,
                         strides=[1, 2, 2, 1],
                         padding="VALID")

这个比较的结果是：

Mean of Abs Diff: 1.6035047067930464e-08

这告诉我的是，如果我能够以某种方式将默认填充行为从 Tensorflow 复制到 PyTorch 中，那么我的结果将是相似的。

This question 检查了 TensorFlow 中的填充行为。 TF documentation explains how padding is added for "SAME" convolutions.我在写这个问题时发现了这些链接。

现在我知道了 TensorFlow 的填充策略，我可以在 PyTorch 中实现它了。

【问题讨论】：

tensorflow SAME 填充可能会在右侧填充比左侧更多的像素。因此，对于 10x10 的图像，内核为 3x3，步幅为 2，tensorflow 只会在右侧和底部填充一列，这与 torch padding=1 不同。

【参考方案1】：

为了复制该行为，填充大小按照 Tensorflow 文档中的说明进行计算。在这里，我通过设置 stride=2 并填充 PyTorch 输入来测试填充行为。

import tensorflow as tf
import numpy as np
import torch
import torch.nn.functional as F


np.random.seed(0)
sess = tf.Session()

# Create random weights and input
weights = torch.empty(3, 3, 3, 8)
torch.nn.init.constant_(weights, 5e-2)
x = np.random.randn(1, 3, 10, 10)

weights_tf = tf.convert_to_tensor(weights.numpy(), dtype=tf.float32)
weights_torch = torch.Tensor(weights.permute((3, 2, 0, 1)))

# Tensorflow padding behavior. Assuming that kH == kW to keep this simple.
stride = 2
if x.shape[2] % stride == 0:
    pad = max(weights.shape[0] - stride, 0)
else:
    pad = max(weights.shape[0] - (x.shape[2] % stride), 0)

if pad % 2 == 0:
    pad_val = pad // 2
    padding = (pad_val, pad_val, pad_val, pad_val)
else:
    pad_val_start = pad // 2
    pad_val_end = pad - pad_val_start
    padding = (pad_val_start, pad_val_end, pad_val_start, pad_val_end)

x_tf = tf.convert_to_tensor(x.transpose((0, 2, 3, 1)), dtype=tf.float32)
x_torch = torch.Tensor(x)
x_torch = F.pad(x_torch, padding, "constant", 0)

# TF Conv2D
tf_conv2d = tf.nn.conv2d(x_tf,
                         weights_tf,
                         strides=[1, stride, stride, 1],
                         padding="SAME")

# PyTorch Conv2D
torch_conv2d = F.conv2d(x_torch, weights_torch, padding=0, stride=stride)

sess.run(tf.global_variables_initializer())
tf_result = sess.run(tf_conv2d)

diff = np.mean(np.abs(tf_result.transpose((0, 3, 1, 2)) - torch_conv2d.detach().numpy()))
print('Mean of Abs Diff: 0'.format(diff))

输出是：

Mean of Abs Diff: 2.2477470551507395e-08

当我开始写这个问题时，我不太清楚为什么会发生这种情况，但是阅读一下很快就澄清了这一点。我希望这个例子可以帮助其他人。