pytorch学习笔记：反向传播

Posted 2021-06-05 Z|Star

关于前馈和反相传播的原理，下面这张实例非常清楚：

import torch
import matplotlib.pyplot as plt

x_data = [1.0,2.0,3.0]
y_data = [2.0,4.0,6.0]

w = torch.Tensor([1.0])
w.requires_grad = True # 需要计算梯度

def forward(x):
    return x * w

def loss(x, y):
    y_pred = forward(x)
    return (y_pred - y) ** 2

print("predict (before training)", 4, forward(4).item())


for epoch in range(100):
    for x, y in zip(x_data, y_data):
        l = loss(x, y)
        l.backward()
        print('\\tgrad:', x, y, w.grad.item())
        w.data = w.data - 0.01 * w.grad.data

        w.grad.data.zero_()  #每一步传完，梯度清零很重要

    print("progress:", epoch, l.item())

print("predict (after training)", 4, forward(4).item())

细节提要：
Tensor张量可以视作数据结构：数据data + 梯度grad（grad也是张量）
核心代码：l.backward()反向传播，程序自动求出所有需要的梯度
w.grad.data.zero_()每步做完清零很重要，否则影响下一次求的梯度