Pytorch CIFAR10图像分类自定义网络篇

Posted 2021-08-30 Real&Love

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Pytorch CIFAR10图像分类自定义网络篇相关的知识，希望对你有一定的参考价值。

Pytorch CIFAR10图像分类自定义网络篇

文章目录

Pytorch CIFAR10图像分类自定义网络篇

这里贴一下汇总篇：汇总篇

4.自定义网络

从网上查了很多关于神经网络的资料，无疑讨论最多的就是网络结构和参数设置，就随便弄了以下的神经网络

1.使用3*3的卷积核

2.使用初始化Xavier

3.使用BN层，减少Dropout使用

4.使用带动量的SGD，或许也可以尝试Adam

5.默认使用ReLU（），或许可以尝试PReLU()

6.batch_size调整为2^n，一般去64,128

7.学习率大小为:0.1->0.01->0.001

首先我们还是得判断是否可以利用GPU，因为GPU的速度可能会比我们用CPU的速度快20-50倍左右，特别是对卷积神经网络来说，更是提升特别明显。

device = 'cuda' if torch.cuda.is_available() else 'cpu'

#定义网络
class Mynet(nn.Module):# nn.Module是所有神经网络的基类，我们自己定义任何神经网络，都要继承nn.Module
    def __init__(self, num_classes=10):
        super(Mynet, self).__init__()
        self.features = nn.Sequential(
            nn.Conv2d(3,64,kernel_size=3,padding=1),
            nn.BatchNorm2d(64),
            nn.ReLU(True),
            
            nn.Conv2d(64,128,kernel_size=3,padding=1),
            nn.BatchNorm2d(128),
            nn.ReLU(True),
            nn.MaxPool2d(kernel_size=2,padding=1),
            
            nn.Conv2d(128,64,kernel_size=3,padding=1),
            nn.BatchNorm2d(64),
            nn.ReLU(True),
            
            nn.Conv2d(64,32,kernel_size=3,padding=1),
            nn.BatchNorm2d(32),
            nn.ReLU(True),
            nn.MaxPool2d(kernel_size=2,padding=1)
            )
        self.classifier = nn.Sequential(
            nn.Linear(32*9*9,2048),
            nn.ReLU(True),
            nn.Linear(2048, num_classes),
            )

 
    def forward(self, x):
        out = self.features(x) 
#         print(out.shape)
        out = out.view(out.size(0), -1)
#         print(out.shape)
        out = self.classifier(out)
#         print(out.shape)
        return out
net = Mynet().to(device)

summary(net,(3,32,32))

----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
            Conv2d-1           [-1, 64, 32, 32]           1,792
       BatchNorm2d-2           [-1, 64, 32, 32]             128
              ReLU-3           [-1, 64, 32, 32]               0
            Conv2d-4          [-1, 128, 32, 32]          73,856
       BatchNorm2d-5          [-1, 128, 32, 32]             256
              ReLU-6          [-1, 128, 32, 32]               0
         MaxPool2d-7          [-1, 128, 17, 17]               0
            Conv2d-8           [-1, 64, 17, 17]          73,792
       BatchNorm2d-9           [-1, 64, 17, 17]             128
             ReLU-10           [-1, 64, 17, 17]               0
           Conv2d-11           [-1, 32, 17, 17]          18,464
      BatchNorm2d-12           [-1, 32, 17, 17]              64
             ReLU-13           [-1, 32, 17, 17]               0
        MaxPool2d-14             [-1, 32, 9, 9]               0
           Linear-15                 [-1, 2048]       5,310,464
             ReLU-16                 [-1, 2048]               0
           Linear-17                   [-1, 10]          20,490
================================================================
Total params: 5,499,434
Trainable params: 5,499,434
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.01
Forward/backward pass size (MB): 5.47
Params size (MB): 20.98
Estimated Total Size (MB): 26.46
----------------------------------------------------------------

首先从我们summary可以看到，我们定义的模型的参数大概是5 millions，我们输入的是（batch，3，32，32）的张量，并且这里也能看到每一层后我们的图像输出大小的变化，最后输出10个参数，再通过softmax函数就可以得到我们每个类别的概率了。

我们也可以打印出我们的模型观察一下

Mynet(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (4): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (5): ReLU(inplace=True)
    (6): MaxPool2d(kernel_size=2, stride=2, padding=1, dilation=1, ceil_mode=False)
    (7): Conv2d(128, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (8): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (9): ReLU(inplace=True)
    (10): Conv2d(64, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (12): ReLU(inplace=True)
    (13): MaxPool2d(kernel_size=2, stride=2, padding=1, dilation=1, ceil_mode=False)
  )
  (classifier): Sequential(
    (0): Linear(in_features=2592, out_features=2048, bias=True)
    (1): ReLU(inplace=True)
    (2): Linear(in_features=2048, out_features=10, bias=True)
  )
)

如果你的电脑有多个GPU，这段代码可以利用GPU进行并行计算，加快运算速度

net =Mynet().to(device)
if device == 'cuda':
    net = nn.DataParallel(net)
    # 当计算图不会改变的时候（每次输入形状相同，模型不改变）的情况下可以提高性能，反之则降低性能
    torch.backends.cudnn.benchmark = True

5. 定义损失函数和优化器

pytorch将深度学习中常用的优化方法全部封装在torch.optim之中，所有的优化方法都是继承基类optim.Optimizier
损失函数是封装在神经网络工具箱nn中的,包含很多损失函数

这里我使用的是SGD + momentum算法，并且我们损失函数定义为交叉熵函数，除此之外学习策略定义为动态更新学习率，如果5次迭代后，训练的损失并没有下降，那么我们便会更改学习率，会变为原来的0.5倍，最小降低到0.00001

如果想更加了解优化器和学习率策略的话，可以参考以下资料

这里决定迭代20次

import torch.optim as optim
optimizer = optim.SGD(net.parameters(), lr=1e-1, momentum=0.9, weight_decay=5e-4)
criterion = nn.CrossEntropyLoss()
scheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer, 'min', factor=0.5 ,patience = 5,min_lr = 0.000001) # 动态更新学习率
# scheduler = optim.lr_scheduler.MultiStepLR(optimizer, milestones=[75, 150], gamma=0.5)
import time
epoch = 20

6. 训练

首先定义模型保存的位置

import os
if not os.path.exists('./model'):
    os.makedirs('./model')
else:
    print('文件已存在')
save_path = './model/Mynet.pth'

我定义了一个train函数，在train函数中进行一个训练，并保存我们训练后的模型

from utils import train
from utils import plot_history
Acc, Loss, Lr = train(net, trainloader, testloader, epoch, optimizer, criterion, scheduler, save_path, verbose = True)

Epoch [  1/ 20]  Train Loss:1.498549  Train Acc:45.00% Test Loss:1.384653  Test Acc:50.54%  Learning Rate:0.100000	Time 00:43
Epoch [  2/ 20]  Train Loss:1.059985  Train Acc:62.00% Test Loss:1.016556  Test Acc:63.80%  Learning Rate:0.100000	Time 00:40
Epoch [  3/ 20]  Train Loss:0.874394  Train Acc:68.95% Test Loss:0.899891  Test Acc:68.67%  Learning Rate:0.100000	Time 00:42
Epoch [  4/ 20]  Train Loss:0.777563  Train Acc:72.65% Test Loss:0.867772  Test Acc:69.60%  Learning Rate:0.100000	Time 00:44
Epoch [  5/ 20]  Train Loss:0.699190  Train Acc:75.54% Test Loss:0.812787  Test Acc:71.54%  Learning Rate:0.100000	Time 00:42
Epoch [  6/ 20]  Train Loss:0.657028  Train Acc:77.06% Test Loss:0.847193  Test Acc:70.65%  Learning Rate:0.100000	Time 00:42
Epoch [  7/ 20]  Train Loss:0.625934  Train Acc:78.05% Test Loss:0.714590  Test Acc:75.08%  Learning Rate:0.100000	Time 00:43
Epoch [  8/ 20]  Train Loss:0.594711  Train Acc:79.31% Test Loss:0.989479  Test Acc:68.00%  Learning Rate:0.100000	Time 00:42
Epoch [  9/ 20]  Train Loss:0.576213  Train Acc:79.96% Test Loss:0.836162  Test Acc:72.17%  Learning Rate:0.100000	Time 00:41
Epoch [ 10/ 20]  Train Loss:0.559027  Train Acc:80.56% Test Loss:0.713146  Test Acc:75.34%  Learning Rate:0.100000	Time 00:41
Epoch [ 11/ 20]  Train Loss:0.535767  Train Acc:81.35% Test Loss:0.774732  Test Acc:75.33%  Learning Rate:0.100000	Time 00:39
Epoch [ 12/ 20]  Train Loss:0.521346  Train Acc:81.88% Test Loss:0.624320  Test Acc:79.46%  Learning Rate:0.100000	Time 00:40
Epoch [ 13/ 20]  Train Loss:0.504253  Train Acc:82.64% Test Loss:0.855251  Test Acc:71.86%  Learning Rate:0.100000	Time 00:40
Epoch [ 14/ 20]  Train Loss:0.499133  Train Acc:82.75% Test Loss:0.677991  Test Acc:76.81%  Learning Rate:0.100000	Tim以上是关于Pytorch CIFAR10图像分类 自定义网络篇的主要内容，如果未能解决你的问题，请参考以下文章 
 Pytorch CIFAR10图像分类 ZFNet篇
 Pytorch CIFAR10图像分类 ZFNet篇
 Pytorch CIFAR10图像分类 ZFNet篇
 Pytorch CIFAR10图像分类 ZFNet篇
 Pytorch CIFAR10图像分类 ZFNet篇
 Pytorch CIFAR10图像分类 ResNet篇

Pytorch CIFAR10图像分类 自定义网络篇

Pytorch CIFAR10图像分类 自定义网络篇

文章目录

4.自定义网络

5. 定义损失函数和优化器

6. 训练

Pytorch CIFAR10图像分类自定义网络篇

Pytorch CIFAR10图像分类自定义网络篇