Paddle 和 PyTorch 中 nn.functional.linear 权重转置导致计算不同

Posted 2023-01-27 氵文大师

我的 Paddle 版本 和 PyTorch 版本：

>>> paddle.__version__
'2.4.0'
>>> torch.__version__
'1.13.1+cu117'

paddle中的 linear 计算公式：
$$Out=XW+b$$

而 Torch 中的计算公式：
$$y=x{A}^T+b$$

torch 中的权重会经过转置在做计算，以下是一个 demo

import paddle
import torch
import numpy as np
import paddle.nn.functional as pF
import torch.nn.functional as tF

np.random.seed(1107)

w = np.random.randn(3, 3)
b = np.random.randn(3,)
x = np.random.randn(2, 3)

w_p = paddle.to_tensor(w)
b_p = paddle.to_tensor(b)
x_p = paddle.to_tensor(x)

w_t = torch.tensor(w)
b_t = torch.tensor(b)
x_t = torch.tensor(x)

y_np = np.matmul(x, w) + b
y_p = pF.linear(x_p, w_p, b_p)
y_t = tF.linear(x_t, w_t.T, b_t)

print(y_np)
print(y_p)
print(y_t)

[[ 0.62959419 -1.8686625   1.47863036]
 [ 1.31505431 -0.37713307 -1.04742683]]
 
Tensor(shape=[2, 3], dtype=float64, place=Place(cpu), stop_gradient=True,
       [[ 0.62959419, -1.86866250,  1.47863036],
        [ 1.31505431, -0.37713307, -1.04742683]])
        
tensor([[ 0.6296, -1.8687,  1.4786],
        [ 1.3151, -0.3771, -1.0474]], dtype=torch.float64)