加载Pytorch中的预训练模型及部分结构的导入

Posted jiangkejie

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了加载Pytorch中的预训练模型及部分结构的导入相关的知识,希望对你有一定的参考价值。

torchvision.model

model子包中包含了用于处理不同任务的经典模型的定义,包括:图像分类、像素级语义分割、对象检测、实例分割、人员关键点检测和视频分类。

图像分类:

技术图片

语义分割:

技术图片

 对象检测、实例分割和人员关键点检测:

技术图片

 视频分类:

  • ResNet 3D
  • ResNet Mixed Convolution
  • ResNet (2+1)D

 模型的导入

你可以通过调用构造函数来构造一个带有随机权重的模型(需要重新训练):

import torchvision.models as models
resnet18 = models.resnet18()
alexnet = models.alexnet()
vgg16 = models.vgg16()
squeezenet = models.squeezenet1_0()
densenet = models.densenet161()
inception = models.inception_v3()
googlenet = models.googlenet()
shufflenet = models.shufflenet_v2_x1_0()
mobilenet = models.mobilenet_v2()
resnext50_32x4d = models.resnext50_32x4d()
wide_resnet50_2 = models.wide_resnet50_2()
mnasnet = models.mnasnet1_0()

导入预训练好的模型:

我们使用PyTorch torch.utils.model_zoo提供预训练的模型。这些可以通过传递pretrained=True来构造:

import torchvision.models as models
resnet18 = models.resnet18(pretrained=True)
alexnet = models.alexnet(pretrained=True)
squeezenet = models.squeezenet1_0(pretrained=True)
vgg16 = models.vgg16(pretrained=True)
densenet = models.densenet161(pretrained=True)
inception = models.inception_v3(pretrained=True)
googlenet = models.googlenet(pretrained=True)
shufflenet = models.shufflenet_v2_x1_0(pretrained=True)
mobilenet = models.mobilenet_v2(pretrained=True)
resnext50_32x4d = models.resnext50_32x4d(pretrained=True)
wide_resnet50_2 = models.wide_resnet50_2(pretrained=True)
mnasnet = models.mnasnet1_0(pretrained=True)

导入预训练模型的部分结构

有时候,我们想利用一些经典模型的架构作为特征提取器,仅想导入部分结构,而不是整个模型。

以densenet121为例,例如仅利用denseblock1及之前的结构+Transition
首先检查densenet121的结构:

import torchvision.models as torch_models
dense = torch_models.densenet121(pretrained=True)

dense.features

输出:

技术图片
  1 Sequential(
  2   (conv0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  3   (norm0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  4   (relu0): ReLU(inplace=True)
  5   (pool0): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
  6   (denseblock1): _DenseBlock(
  7     (denselayer1): _DenseLayer(
  8       (norm1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  9       (relu1): ReLU(inplace=True)
 10       (conv1): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
 11       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 12       (relu2): ReLU(inplace=True)
 13       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
 14     )
 15     (denselayer2): _DenseLayer(
 16       (norm1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 17       (relu1): ReLU(inplace=True)
 18       (conv1): Conv2d(96, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
 19       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 20       (relu2): ReLU(inplace=True)
 21       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
 22     )
 23     (denselayer3): _DenseLayer(
 24       (norm1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 25       (relu1): ReLU(inplace=True)
 26       (conv1): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
 27       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 28       (relu2): ReLU(inplace=True)
 29       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
 30     )
 31     (denselayer4): _DenseLayer(
 32       (norm1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 33       (relu1): ReLU(inplace=True)
 34       (conv1): Conv2d(160, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
 35       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 36       (relu2): ReLU(inplace=True)
 37       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
 38     )
 39     (denselayer5): _DenseLayer(
 40       (norm1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 41       (relu1): ReLU(inplace=True)
 42       (conv1): Conv2d(192, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
 43       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 44       (relu2): ReLU(inplace=True)
 45       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
 46     )
 47     (denselayer6): _DenseLayer(
 48       (norm1): BatchNorm2d(224, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 49       (relu1): ReLU(inplace=True)
 50       (conv1): Conv2d(224, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
 51       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 52       (relu2): ReLU(inplace=True)
 53       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
 54     )
 55   )
 56   (transition1): _Transition(
 57     (norm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 58     (relu): ReLU(inplace=True)
 59     (conv): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
 60     (pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
 61   )
 62   (denseblock2): _DenseBlock(
 63     (denselayer1): _DenseLayer(
 64       (norm1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 65       (relu1): ReLU(inplace=True)
 66       (conv1): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
 67       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 68       (relu2): ReLU(inplace=True)
 69       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
 70     )
 71     (denselayer2): _DenseLayer(
 72       (norm1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 73       (relu1): ReLU(inplace=True)
 74       (conv1): Conv2d(160, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
 75       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 76       (relu2): ReLU(inplace=True)
 77       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
 78     )
 79     (denselayer3): _DenseLayer(
 80       (norm1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 81       (relu1): ReLU(inplace=True)
 82       (conv1): Conv2d(192, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
 83       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 84       (relu2): ReLU(inplace=True)
 85       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
 86     )
 87     (denselayer4): _DenseLayer(
 88       (norm1): BatchNorm2d(224, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 89       (relu1): ReLU(inplace=True)
 90       (conv1): Conv2d(224, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
 91       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 92       (relu2): ReLU(inplace=True)
 93       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
 94     )
 95     (denselayer5): _DenseLayer(
 96       (norm1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 97       (relu1): ReLU(inplace=True)
 98       (conv1): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
 99       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
100       (relu2): ReLU(inplace=True)
101       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
102     )
103     (denselayer6): _DenseLayer(
104       (norm1): BatchNorm2d(288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
105       (relu1): ReLU(inplace=True)
106       (conv1): Conv2d(288, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
107       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
108       (relu2): ReLU(inplace=True)
109       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
110     )
111     (denselayer7): _DenseLayer(
112       (norm1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
113       (relu1): ReLU(inplace=True)
114       (conv1): Conv2d(320, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
115       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
116       (relu2): ReLU(inplace=True)
117       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
118     )
119     (denselayer8): _DenseLayer(
120       (norm1): BatchNorm2d(352, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
121       (relu1): ReLU(inplace=True)
122       (conv1): Conv2d(352, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
123       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
124       (relu2): ReLU(inplace=True)
125       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
126     )
127     (denselayer9): _DenseLayer(
128       (norm1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
129       (relu1): ReLU(inplace=True)
130       (conv1): Conv2d(384, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
131       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
132       (relu2): ReLU(inplace=True)
133       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
134     )
135     (denselayer10): _DenseLayer(
136       (norm1): BatchNorm2d(416, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
137       (relu1): ReLU(inplace=True)
138       (conv1): Conv2d(416, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
139       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
140       (relu2): ReLU(inplace=True)
141       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
142     )
143     (denselayer11): _DenseLayer(
144       (norm1): BatchNorm2d(448, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
145       (relu1): ReLU(inplace=True)
146       (conv1): Conv2d(448, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
147       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
148       (relu2): ReLU(inplace=True)
149       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
150     )
151     (denselayer12): _DenseLayer(
152       (norm1): BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
153       (relu1): ReLU(inplace=True)
154       (conv1): Conv2d(480, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
155       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
156       (relu2): ReLU(inplace=True)
157       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
158     )
159   )
160   (transition2): _Transition(
161     (norm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
162     (relu): ReLU(inplace=True)
163     (conv): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
164     (pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
165   )
166   (denseblock3): _DenseBlock(
167     (denselayer1): _DenseLayer(
168       (norm1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
169       (relu1): ReLU(inplace=True)
170       (conv1): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
171       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
172       (relu2): ReLU(inplace=True)
173       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
174     )
175     (denselayer2): _DenseLayer(
176       (norm1): BatchNorm2d(288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
177       (relu1): ReLU(inplace=True)
178       (conv1): Conv2d(288, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
179       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
180       (relu2): ReLU(inplace=True)
181       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
182     )
183     (denselayer3): _DenseLayer(
184       (norm1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
185       (relu1): ReLU(inplace=True)
186       (conv1): Conv2d(320, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
187       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
188       (relu2): ReLU(inplace=True)
189       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
190     )
191     (denselayer4): _DenseLayer(
192       (norm1): BatchNorm2d(352, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
193       (relu1): ReLU(inplace=True)
194       (conv1): Conv2d(352, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
195       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
196       (relu2): ReLU(inplace=True)
197       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
198     )
199     (denselayer5): _DenseLayer(
200       (norm1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
201       (relu1): ReLU(inplace=True)
202       (conv1): Conv2d(384, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
203       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
204       (relu2): ReLU(inplace=True)
205       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
206     )
207     (denselayer6): _DenseLayer(
208       (norm1): BatchNorm2d(416, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
209       (relu1): ReLU(inplace=True)
210       (conv1): Conv2d(416, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
211       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
212       (relu2): ReLU(inplace=True)
213       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
214     )
215     (denselayer7): _DenseLayer(
216       (norm1): BatchNorm2d(448, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
217       (relu1): ReLU(inplace=True)
218       (conv1): Conv2d(448, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
219       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
220       (relu2): ReLU(inplace=True)
221       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
222     )
223     (denselayer8): _DenseLayer(
224       (norm1): BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
225       (relu1): ReLU(inplace=True)
226       (conv1): Conv2d(480, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
227       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
228       (relu2): ReLU(inplace=True)
229       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
230     )
231     (denselayer9): _DenseLayer(
232       (norm1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
233       (relu1): ReLU(inplace=True)
234       (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
235       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
236       (relu2): ReLU(inplace=True)
237       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
238     )
239     (denselayer10): _DenseLayer(
240       (norm1): BatchNorm2d(544, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
241       (relu1): ReLU(inplace=True)
242       (conv1): Conv2d(544, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
243       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
244       (relu2): ReLU(inplace=True)
245       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
246     )
247     (denselayer11): _DenseLayer(
248       (norm1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
249       (relu1): ReLU(inplace=True)
250       (conv1): Conv2d(576, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
251       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
252       (relu2): ReLU(inplace=True)
253       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
254     )
255     (denselayer12): _DenseLayer(
256       (norm1): BatchNorm2d(608, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
257       (relu1): ReLU(inplace=True)
258       (conv1): Conv2d(608, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
259       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
260       (relu2): ReLU(inplace=True)
261       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
262     )
263     (denselayer13): _DenseLayer(
264       (norm1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
265       (relu1): ReLU(inplace=True)
266       (conv1): Conv2d(640, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
267       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
268       (relu2): ReLU(inplace=True)
269       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
270     )
271     (denselayer14): _DenseLayer(
272       (norm1): BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
273       (relu1): ReLU(inplace=True)
274       (conv1): Conv2d(672, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
275       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
276       (relu2): ReLU(inplace=True)
277       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
278     )
279     (denselayer15): _DenseLayer(
280       (norm1): BatchNorm2d(704, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
281       (relu1): ReLU(inplace=True)
282       (conv1): Conv2d(704, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
283       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
284       (relu2): ReLU(inplace=True)
285       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
286     )
287     (denselayer16): _DenseLayer(
288       (norm1): BatchNorm2d(736, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
289       (relu1): ReLU(inplace=True)
290       (conv1): Conv2d(736, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
291       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
292       (relu2): ReLU(inplace=True)
293       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
294     )
295     (denselayer17): _DenseLayer(
296       (norm1): BatchNorm2d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
297       (relu1): ReLU(inplace=True)
298       (conv1): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
299       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
300       (relu2): ReLU(inplace=True)
301       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
302     )
303     (denselayer18): _DenseLayer(
304       (norm1): BatchNorm2d(800, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
305       (relu1): ReLU(inplace=True)
306       (conv1): Conv2d(800, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
307       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
308       (relu2): ReLU(inplace=True)
309       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
310     )
311     (denselayer19): _DenseLayer(
312       (norm1): BatchNorm2d(832, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
313       (relu1): ReLU(inplace=True)
314       (conv1): Conv2d(832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
315       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
316       (relu2): ReLU(inplace=True)
317       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
318     )
319     (denselayer20): _DenseLayer(
320       (norm1): BatchNorm2d(864, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
321       (relu1): ReLU(inplace=True)
322       (conv1): Conv2d(864, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
323       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
324       (relu2): ReLU(inplace=True)
325       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
326     )
327     (denselayer21): _DenseLayer(
328       (norm1): BatchNorm2d(896, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
329       (relu1): ReLU(inplace=True)
330       (conv1): Conv2d(896, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
331       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
332       (relu2): ReLU(inplace=True)
333       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
334     )
335     (denselayer22): _DenseLayer(
336       (norm1): BatchNorm2d(928, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
337       (relu1): ReLU(inplace=True)
338       (conv1): Conv2d(928, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
339       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
340       (relu2): ReLU(inplace=True)
341       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
342     )
343     (denselayer23): _DenseLayer(
344       (norm1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
345       (relu1): ReLU(inplace=True)
346       (conv1): Conv2d(960, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
347       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
348       (relu2): ReLU(inplace=True)
349       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
350     )
351     (denselayer24): _DenseLayer(
352       (norm1): BatchNorm2d(992, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
353       (relu1): ReLU(inplace=True)
354       (conv1): Conv2d(992, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
355       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
356       (relu2): ReLU(inplace=True)
357       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
358     )
359   )
360   (transition3): _Transition(
361     (norm): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
362     (relu): ReLU(inplace=True)
363     (conv): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
364     (pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
365   )
366   (denseblock4): _DenseBlock(
367     (denselayer1): _DenseLayer(
368       (norm1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
369       (relu1): ReLU(inplace=True)
370       (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
371       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
372       (relu2): ReLU(inplace=True)
373       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
374     )
375     (denselayer2): _DenseLayer(
376       (norm1): BatchNorm2d(544, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
377       (relu1): ReLU(inplace=True)
378       (conv1): Conv2d(544, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
379       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
380       (relu2): ReLU(inplace=True)
381       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
382     )
383     (denselayer3): _DenseLayer(
384       (norm1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
385       (relu1): ReLU(inplace=True)
386       (conv1): Conv2d(576, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
387       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
388       (relu2): ReLU(inplace=True)
389       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
390     )
391     (denselayer4): _DenseLayer(
392       (norm1): BatchNorm2d(608, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
393       (relu1): ReLU(inplace=True)
394       (conv1): Conv2d(608, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
395       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
396       (relu2): ReLU(inplace=True)
397       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
398     )
399     (denselayer5): _DenseLayer(
400       (norm1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
401       (relu1): ReLU(inplace=True)
402       (conv1): Conv2d(640, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
403       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
404       (relu2): ReLU(inplace=True)
405       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
406     )
407     (denselayer6): _DenseLayer(
408       (norm1): BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
409       (relu1): ReLU(inplace=True)
410       (conv1): Conv2d(672, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
411       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
412       (relu2): ReLU(inplace=True)
413       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
414     )
415     (denselayer7): _DenseLayer(
416       (norm1): BatchNorm2d(704, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
417       (relu1): ReLU(inplace=True)
418       (conv1): Conv2d(704, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
419       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
420       (relu2): ReLU(inplace=True)
421       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
422     )
423     (denselayer8): _DenseLayer(
424       (norm1): BatchNorm2d(736, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
425       (relu1): ReLU(inplace=True)
426       (conv1): Conv2d(736, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
427       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
428       (relu2): ReLU(inplace=True)
429       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
430     )
431     (denselayer9): _DenseLayer(
432       (norm1): BatchNorm2d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
433       (relu1): ReLU(inplace=True)
434       (conv1): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
435       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
436       (relu2): ReLU(inplace=True)
437       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
438     )
439     (denselayer10): _DenseLayer(
440       (norm1): BatchNorm2d(800, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
441       (relu1): ReLU(inplace=True)
442       (conv1): Conv2d(800, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
443       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
444       (relu2): ReLU(inplace=True)
445       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
446     )
447     (denselayer11): _DenseLayer(
448       (norm1): BatchNorm2d(832, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
449       (relu1): ReLU(inplace=True)
450       (conv1): Conv2d(832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
451       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
452       (relu2): ReLU(inplace=True)
453       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
454     )
455     (denselayer12): _DenseLayer(
456       (norm1): BatchNorm2d(864, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
457       (relu1): ReLU(inplace=True)
458       (conv1): Conv2d(864, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
459       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
460       (relu2): ReLU(inplace=True)
461       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
462     )
463     (denselayer13): _DenseLayer(
464       (norm1): BatchNorm2d(896, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
465       (relu1): ReLU(inplace=True)
466       (conv1): Conv2d(896, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
467       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
468       (relu2): ReLU(inplace=True)
469       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
470     )
471     (denselayer14): _DenseLayer(
472       (norm1): BatchNorm2d(928, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
473       (relu1): ReLU(inplace=True)
474       (conv1): Conv2d(928, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
475       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
476       (relu2): ReLU(inplace=True)
477       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
478     )
479     (denselayer15): _DenseLayer(
480       (norm1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
481       (relu1): ReLU(inplace=True)
482       (conv1): Conv2d(960, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
483       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
484       (relu2): ReLU(inplace=True)
485       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
486     )
487     (denselayer16): _DenseLayer(
488       (norm1): BatchNorm2d(992, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
489       (relu1): ReLU(inplace=True)
490       (conv1): Conv2d(992, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
491       (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
492       (relu2): ReLU(inplace=True)
493       (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
494     )
495   )
496   (norm5): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
497 )
View Code

 关于densenet的Pytorch实现源码解析参考【pytorch torchvision源码解读系列—5】DenseNet,关于Densenet参考DenseNet

技术图片

 检查densenet的结构:

技术图片
 1 dense.features[0]
 2 
 3 Out[10]: Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
 4 
 5 dense.features[1]
 6 Out[11]: BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
 7 
 8 dense.features[2]
 9 Out[12]: ReLU(inplace=True)
10 
11 dense.features[3]
12 Out[13]: MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
13 
14 dense.features[4]
15 Out[14]: 
16 _DenseBlock(
17   (denselayer1): _DenseLayer(
18     (norm1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
19     (relu1): ReLU(inplace=True)
20     (conv1): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
21     (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
22     (relu2): ReLU(inplace=True)
23     (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
24   )
25   (denselayer2): _DenseLayer(
26     (norm1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
27     (relu1): ReLU(inplace=True)
28     (conv1): Conv2d(96, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
29     (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
30     (relu2): ReLU(inplace=True)
31     (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
32   )
33   (denselayer3): _DenseLayer(
34     (norm1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
35     (relu1): ReLU(inplace=True)
36     (conv1): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
37     (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
38     (relu2): ReLU(inplace=True)
39     (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
40   )
41   (denselayer4): _DenseLayer(
42     (norm1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
43     (relu1): ReLU(inplace=True)
44     (conv1): Conv2d(160, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
45     (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
46     (relu2): ReLU(inplace=True)
47     (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
48   )
49   (denselayer5): _DenseLayer(
50     (norm1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
51     (relu1): ReLU(inplace=True)
52     (conv1): Conv2d(192, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
53     (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
54     (relu2): ReLU(inplace=True)
55     (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
56   )
57   (denselayer6): _DenseLayer(
58     (norm1): BatchNorm2d(224, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
59     (relu1): ReLU(inplace=True)
60     (conv1): Conv2d(224, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
61     (norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
62     (relu2): ReLU(inplace=True)
63     (conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
64   )
65 )
66 
67 
68 dense.features[5]
69 Out[15]: 
70 _Transition(
71   (norm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
72   (relu): ReLU(inplace=True)
73   (conv): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
74   (pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
75 )
View Code

 调用部分结构即可,例如:

 1 class mydensenet121(nn.Module): # [N, 3, 224, 224] --> [N, 128, 56, 56]
 2     def __init__(self):
 3         super(mydensenet121, self).__init__()
 4         densenet_pretrained = torch_models.densenet121(pretrained=True)
 5         self.features = densenet_pretrained.features[:5] # 预处理层+denseblock1(denselayer1:6)
 6         self.bn_relu_1x1conv = densenet_pretrained.features[5][:3]  # 这里报错?
 7 
 8 
 9     def forward(self, x):
10         out = self.features(x)
11         out = self.bn_relu_1x1conv(out)
12         return out

 

1 TypeError: __init__() missing 1 required positional argument: num_output_features

但是分别调用是没问题的,不知道为啥?

技术图片

 

以上是关于加载Pytorch中的预训练模型及部分结构的导入的主要内容,如果未能解决你的问题,请参考以下文章

Pytorch 中的预训练模型

pytorch加载内置模型、修改网络结构及加载预训练参数

PyTorch 的预训练,是时候学习一下了

如果我们扩展或减少同一模型的层,我们仍然可以从 Pytorch 中的预训练模型进行训练吗?

Pytorch——GPT-2 预训练模型及文本生成

预训练模型的加载机理pytorch版