[十九]深度学习Pytorch-可视化工具TensorBoard
Posted yanzhiwen2
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了[十九]深度学习Pytorch-可视化工具TensorBoard相关的知识，希望对你有一定的参考价值。
0. 往期内容
[二]深度学习Pytorch-张量的操作：拼接、切分、索引和变换
[七]深度学习Pytorch-DataLoader与Dataset(含人民币二分类实战)
[八]深度学习Pytorch-图像预处理transforms
[九]深度学习Pytorch-transforms图像增强(剪裁、翻转、旋转)
[十]深度学习Pytorch-transforms图像操作及自定义方法
[十一]深度学习Pytorch-模型创建与nn.Module
[十二]深度学习Pytorch-模型容器与AlexNet构建
[十三]深度学习Pytorch-卷积层(1D/2D/3D卷积、卷积nn.Conv2d、转置卷积nn.ConvTranspose)
[十六]深度学习Pytorch-18种损失函数loss function
[十八]深度学习Pytorch-学习率Learning Rate调整策略
[十九]深度学习Pytorch-可视化工具TensorBoard
深度学习Pytorch-可视化工具TensorBoard
1. TensorBoard简介
2. TensorBoard安装
显示没有past模块时，应该安装future。
3. TensorBoard可视化
命令：tensorboard –logdir=event_file所在的路径
一定要选择数据后才可以下载。
4. SummaryWriter
当设置log_dir时，comment不会起作用。
代码示例：
# ----------------------------------- 0 SummaryWriter -----------------------------------
flag = 0
# flag = 1
if flag:
log_dir = "./train_log/test_log_dir"
# writer = SummaryWriter(log_dir=log_dir, comment='_scalars', filename_suffix="12345678")
writer = SummaryWriter(comment='_scalars', filename_suffix="12345678")
for x in range(100):
writer.add_scalar('y=pow_2_x', 2 ** x, x)
writer.close()
官网示例：
from torch.utils.tensorboard import SummaryWriter
# create a summary writer with automatically generated folder name.
writer = SummaryWriter()
# folder location: runs/May04_22-14-54_s-MacBook-Pro.local/
# create a summary writer using the specified folder name.
writer = SummaryWriter("my_experiment")
# folder location: my_experiment
# create a summary writer with comment appended.
writer = SummaryWriter(comment="LR_0.1_BATCH_16")
# folder location: runs/May04_22-14-54_s-MacBook-Pro.localLR_0.1_BATCH_16/
4.1 & 4.2 add_scalar & add_scalars
add_scalar(tag, scalar_value, global_step=None, walltime=None, new_style=False, double_precision=False)
add_scalars(main_tag, tag_scalar_dict, global_step=None, walltime=None)
代码示例：
# ----------------------------------- 1 scalar and scalars -----------------------------------
flag = 0
# flag = 1
if flag:
max_epoch = 100
writer = SummaryWriter(comment='test_comment', filename_suffix="test_suffix")
for x in range(max_epoch):
writer.add_scalar('y=2x', x * 2, x)
writer.add_scalar('y=pow_2_x', 2 ** x, x)
writer.add_scalars('data/scalar_group', "xsinx": x * np.sin(x),
"xcosx": x * np.cos(x), x)
writer.close()
4.3 add_histogram(tag, values)
add_histogram(tag, values, global_step=None, bins='tensorflow', walltime=None, max_bins=None)
（1）统计的参数通常为权值、偏置、梯度等。
（2）tensorboard可视化：后面的层梯度很大，前面的层梯度很小，考虑是梯度消失，反向传播时梯度逐渐变小。
代码示例：
# ----------------------------------- 2 histogram -----------------------------------
# flag = 0
flag = 1
if flag:
writer = SummaryWriter(comment='test_comment', filename_suffix="test_suffix")
for x in range(2):
np.random.seed(x)
data_union = np.arange(100)
data_normal = np.random.normal(size=1000)
writer.add_histogram('distribution union', data_union, x)
writer.add_histogram('distribution normal', data_normal, x)
plt.subplot(121).hist(data_union, label="union")
plt.subplot(122).hist(data_normal, label="normal")
plt.legend()
plt.show()
writer.close()
4.4 add_image(tag, img_tensor, dataformats=‘CHW’)
add_image(tag, img_tensor, global_step=None, walltime=None, dataformats='CHW')
img_tensor要求是0-255区间。会根据图像数据自动缩放到0-255区间。但是，如果像素值超过1，会认为已经是0-255区间，不再自动执行缩放到0-255区间。
代码示例：
# ----------------------------------- 3 image -----------------------------------
flag = 0
# flag = 1
if flag:
writer = SummaryWriter(comment='test_your_comment', filename_suffix="_test_your_filename_suffix")
# img 1 random
fake_img = torch.randn(3, 512, 512)
writer.add_image("fake_img", fake_img, 1)
time.sleep(1)
# img 2 ones
fake_img = torch.ones(3, 512, 512)
time.sleep(1)
writer.add_image("fake_img", fake_img, 2)
# img 3 1.1
fake_img = torch.ones(3, 512, 512) * 1.1
time.sleep(1)
writer.add_image("fake_img", fake_img, 3)
# img 4 HW
fake_img = torch.rand(512, 512)
writer.add_image("fake_img", fake_img, 4, dataformats="HW")
# img 5 HWC
fake_img = torch.rand(512, 512, 3)
writer.add_image("fake_img", fake_img, 5, dataformats="HWC")
writer.close()
官网示例：
from torch.utils.tensorboard import SummaryWriter
import numpy as np
img = np.zeros((3, 100, 100))
img[0] = np.arange(0, 10000).reshape(100, 100) / 10000
img[1] = 1 - np.arange(0, 10000).reshape(100, 100) / 10000
img_HWC = np.zeros((100, 100, 3))
img_HWC[:, :, 0] = np.arange(0, 10000).reshape(100, 100) / 10000
img_HWC[:, :, 1] = 1 - np.arange(0, 10000).reshape(100, 100) / 10000
writer = SummaryWriter()
writer.add_image('my_image', img, 0)
# If you have non-default dimension setting, set the dataformats argument.
writer.add_image('my_image_HWC', img_HWC, 0, dataformats='HWC')
writer.close()
4.5 make_grid
(1)像素值标准化即0-255区间。
(2)比如有个数据的原始范围是[-1000,2000]，设置range=[-600,500]，那么小于-600的都会设置为-600，大于500的都会设置为500，然后对[-600,500]区间进行标准化为[0,255]。
代码示例：
# ----------------------------------- 4 make_grid -----------------------------------
flag = 0
# flag = 1
if flag:
writer = SummaryWriter(comment='test_your_comment', filename_suffix="_test_your_filename_suffix")
split_dir = os.path.join("..", "..", "data", "rmb_split")
train_dir = os.path.join(split_dir, "train")
# train_dir = "path to your training data"
transform_compose = transforms.Compose([transforms.Resize((32, 64)), transforms.ToTensor()])
train_data = RMBDataset(data_dir=train_dir, transform=transform_compose)
train_loader = DataLoader(dataset=train_data, batch_size=16, shuffle=True)
data_batch, label_batch = next(iter(train_loader)) #取出一个batchsize的数据，data_batch是4D：BCHW
img_grid = vutils.make_grid(data_batch, nrow=4, normalize=True, scale_each=True) #batchsize是16，所以是4行4列，16/4=4
# img_grid = vutils.make_grid(data_batch, nrow=4, normalize=False, scale_each=False)
writer.add_image("input img", img_grid, 0)
writer.close()
4.6 add_graph(model, input_to_model=None)
add_graph(model, input_to_model=None, verbose=False, use_strict_trace=True)
pytorch1.2不行，必须1.3或1.3+
代码示例：
# ----------------------------------- 5 add_graph -----------------------------------
# flag = 0
flag = 1
if flag:
writer = SummaryWriter(comment='test_your_comment', filename_suffix="_test_your_filename_suffix")
# 模型
fake_img = torch.randn(1, 3, 32, 32)
lenet = LeNet(classes=2)
writer.add_graph(lenet, fake_img)
writer.close()
4.7 torchsummary
代码示例：
flag = 1
if flag:
writer = SummaryWriter(comment='test_your_comment', filename_suffix="_test_your_filename_suffix")
# 模型
fake_img = torch.randn(1, 3, 32, 32)
lenet = LeNet(classes=2)
writer.add_graph(lenet, fake_img)
writer.close()
from torchsummary import summary
print(summary(lenet, (3, 32, 32), device="cpu"))
第一个卷积层是6*3*5*5=450，每个卷积核有1个bias，因此是450+1*6=456。
7. 完整代码
loss_acc_weights_grad.py
# -*- coding:utf-8 -*-
"""
@file name : loss_acc_weights_grad.py
@brief : 监控loss, accuracy, weights, gradients
"""
import os
import numpy as np
import torch
import torch.nn as nn
from torch.utils.data import DataLoader
import torchvision.transforms as transforms
from torch.utils.tensorboard import SummaryWriter
import torch.optim as optim
from matplotlib import pyplot as plt
from model.lenet import LeNet
from tools.my_dataset import RMBDataset
from tools.common_tools import set_seed
set_seed() # 设置随机种子
rmb_label = "1": 0, "100": 1
# 参数设置
MAX_EPOCH = 10
BATCH_SIZE = 16
LR = 0.01
log_interval = 10
val_interval = 1
# ============================ step 1/5 数据 ============================
split_dir = os.path.join("..", "..", "data", "rmb_split")
train_dir = os.path.join(split_dir, "train")
valid_dir = os.path.join(split_dir, "valid")
norm_mean = [0.485, 0.456, 0.406]
norm_std = [0.229, 0.224, 0.225]
train_transform = transforms.Compose([
transforms.Resize((32, 32)),
transforms.RandomCrop(32, padding=4),
transforms.RandomGrayscale(p=0.8),
transforms.ToTensor(),
transforms.Normalize(norm_mean, norm_std),
])
valid_transform = transforms.Compose([
transforms.Resize((32, 32)),
transforms.ToTensor(),
transforms.Normalize(norm_mean, norm_std),
])
# 构建MyDataset实例
train_data = RMBDataset(data_dir=train_dir, transform=train_transform)
valid_data = RMBDataset(data_dir=valid_dir, transform=valid_transform)
# 构建DataLoder
train_loader = DataLoader(dataset=train_data, batch_size=BATCH_SIZE, shuffle=True)
valid_loader = DataLoader(dataset=valid_data, batch_size=BATCH_SIZE)
# ============================ step 2/5 模型 ============================
net = LeNet(classes=2)
net.initialize_weights()
# ============================ step 3/5 损失函数 ============================
criterion = nn.CrossEntropyLoss() # 选择损失函数
# ============================ step 4/5 优化器 ============================
optimizer = optim.SGD(net.parameters(), lr=LR, momentum=0.9) # 选择优化器
scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=10, gamma=0.1) # 设置学习率下降策略
# ============================ step 5/5 训练 ============================
train_curve = list()
valid_curve = list()
iter_count = 0
# 构建 SummaryWriter
writer = SummaryWriter(comment='test_your_comment', filename_suffix="_test_your_filename_suffix")
for epoch in range(MAX_EPOCH):
loss_mean = 0.
correct = 0.
total = 0.
net.train()
for i, data in enumerate(train_loader):
iter_count += 1
# forward
inputs, labels = data
outputs = net(inputs)
# backward
optimizer.zero_grad()
loss = criterion(outputs, labels)
loss.backward()
# update weights
optimizer.step()
# 统计分类情况
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).squeeze().sum().numpy()
# 打印训练信息
loss_mean += loss.item()
train_curve.append(loss.item())
if (i+1) % log_interval == 0:
loss_mean = loss_mean / log_interval
print("Training:Epoch[:0>3/:0>3] Iteration[:0>3/:0>3] Loss: :.4f Acc::.2%".format(
epoch, MAX_EPOCH, i+1, len(train_loader), loss_mean, correct / total))
loss_mean = 0.
# 记录数据，保存于event file
writer.add_scalars("Loss", "Train": loss.item(), iter_count) #记录的loss是batchsize的平均值，可能会有振荡，因为batchsize具有随机性。
writer.add_scalars("Accuracy", "Train": correct / total, iter_count)
# 每个epoch，记录梯度，权值
for name, param in net.named_parameters():
writer.add_histogram(name + '_grad', param.grad, epoch)
writer.add_histogram(name + '_data', param, epoch)
scheduler.step() # 更新学习率
# validate the model
if (以上是关于[十九]深度学习Pytorch-可视化工具TensorBoard的主要内容，如果未能解决你的问题，请参考以下文章
深度学习100例 | 第2例：人脸表情识别 - PyTorch实现
深度学习100例 | 第2例：人脸表情识别 - PyTorch实现
深度学习pytorch使用tensorboard可视化实验数据