pytorch 笔记 torch.nn.BatchNorm1d

Posted 2022-07-01 UQI-LIUWJ

1 主要参数介绍

torch.nn.BatchNorm1d(num_features, eps=1e-05)

num_features	需要进行归一化的数据维度，一般等于dim_hid
eps	加到分母上的值，以保证数值稳定性

2 num_feature分析

batch normalization是对一个batch里面的每一个维度分别进行归一化

 

举一个例子：

a=torch.Tensor([[1,1,1],
                [2,-3,5],
                [0.1,-0.3,0.5],
                [1.2,-1.3,1.5]])
f1=torch.nn.BatchNorm1d(3)
f2=torch.nn.BatchNorm1d(4)
print(a,'\\n',f1(a),'\\n',f2(a.T))

'''
tensor([[ 1.0000,  1.0000,  1.0000],
        [ 2.0000, -3.0000,  5.0000],
        [ 0.1000, -0.3000,  0.5000],
        [ 1.2000, -1.3000,  1.5000]]) 
 tensor([[-0.1110,  1.3003, -0.5657],
        [ 1.3685, -1.4372,  1.6971],
        [-1.4425,  0.4106, -0.8485],
        [ 0.1849, -0.2738, -0.2828]], grad_fn=<NativeBatchNormBackward0>) 
 tensor([[ 0.0000e+00,  2.0203e-01,  2.2812e-08,  5.8423e-01],
        [ 0.0000e+00, -1.3132e+00, -1.2247e+00, -1.4075e+00],
        [ 0.0000e+00,  1.1112e+00,  1.2247e+00,  8.2323e-01]],
       grad_fn=<NativeBatchNormBackward0>)
'''

num_features是3，表示每个sample有三个feature，那么就有三组数据分别要进行归一化