深度学习开发框架PyTorch(4)-- 神经网络建模

使用Autograd可实现深度学习模型,但其抽象程度较低,如果用来实现深度学习模型需要编写的代码量极大。torch.nn模块应运而生,它们是专门为深度学习设计的模块。

一、神经网络层

nn.Module是torch.nn的核心数据结构,它是一个抽象的概念,既可以表示神经网络中的某个层(layer),也可以表示一个包含很多层的神经网络。nn.Module的特点是能够自动检测到自身及其子Module的Parameter,并将其作为学习参数。

1、自定义网络层

全连接层又名仿射层,输出y和输入x满足y=Wx+b, W和b是可学习的参数。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# coding:utf-8
'''
自定义全连接层
'''
import torch as t
from torch import nn
from torch.autograd import Variable as V
import warnings; warnings.filterwarnings(action='ignore')

class Linear(nn.Module):
def __init__(self, in_features, out_features):
super(Linear, self).__init__() # 等价于nn.Module.__init__()
# 自定义可学习的参数并封装成Parameter, 它是一种特殊的Variable
self.w = nn.Parameter(t.randn(in_features, out_features))
self.b = nn.Parameter(t.randn(out_features))

def forward(self, x):
x = x.mm(self.w)
return x + self.b.expand_as(x)

input = V(t.randn(5, 4))
print(input)

layer = Linear(4, 1)
output = layer(input)

print(output)

for name, parameter in layer.named_parameters():
print(name, parameter) # w 和 b

接下来用上面定义的全连接层构造一个两层的感知机,采用sigmoid函数作为激活函数。注意,第一个继承自nn.Module的类代表的是一个层,第二个继承自nn.Module的类代表的是一个网络。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
# coding:utf-8
'''
自定义多层感知机
'''
import torch as t
from torch import nn
from torch.autograd import Variable as V
import warnings; warnings.filterwarnings(action='ignore')

class Linear(nn.Module):
def __init__(self, in_features, out_features):
super(Linear, self).__init__() # 等价于nn.Module.__init__()
# 自定义可学习的参数并封装成Parameter, 它是一种特殊的Variable
self.w = nn.Parameter(t.randn(in_features, out_features))
self.b = nn.Parameter(t.randn(out_features))

def forward(self, x):
x = x.mm(self.w)
return x + self.b.expand_as(x)

class Perceptron(nn.Module):
def __init__(self, in_features, hidden_features, out_features):
super(Perceptron, self).__init__()
self.layer1 = Linear(in_features, hidden_features) #使用的是上面定义的Linear函数
self.layer2 = Linear(hidden_features, out_features)

def forward(self, x):
x = self.layer1(x)
x = t.sigmoid(x) #激活函数
return self.layer2(x)

perceptron = Perceptron(3,4,1)

for name, param in perceptron.named_parameters():
print(name, param.size())

2、内置网络层

为方便用户使用,PyTorch实现了神经网络中绝大多数的layer,这些layer都继承于nn.Module,封装了可学习参数parameter,并实现了forward函数,且很多都专门针对GPU运算进行了CuDNN优化,其速度和性能都十分优异。
注意:实际使用中除非需要使用特殊的初始化,否则应尽量不要直接修改内置网络层的属性。

(1)全连接层
1
2
3
4
5
6
7
8
9
10
11
12
import torch as t
import torch.nn as nn

# 样本数为1, 维度为3
input = t.randn(1, 3)
print(input.size())

# 全连接层, 输入节点3, 输出节点1
linear = nn.Linear(3, 1)
out = linear(input)

print(out)
(2)卷积层
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
from PIL import Image
import numpy as np
import matplotlib.pyplot as plt

import torch as t
import torch.nn as nn
from torch.autograd import Variable as V
from torchvision.transforms import ToTensor, ToPILImage

to_tensor = ToTensor() # img -> tensor
to_pil = ToPILImage() # tensor -> img

image = np.array(Image.open('lena.jpg'))

# 样本数=1,通道数=1,图像的shape是256乘256的
input = to_tensor(image).unsqueeze(0)
print(input.size())

# 锐化卷积核
kernel = t.ones(3, 3)/-9
kernel[1][1] = 1
# print(kernel)

# 卷积层1, 输入通道数1, 卷积核个数1, 卷积核规格3乘以3
conv1 = nn.Conv2d(1, 1, kernel_size=(3, 3), stride=1, padding=0, bias=False)
print(conv1(V(input)).size())
# 卷积层2, 输入通道数1, 卷积核个数3, 卷积核规格5乘以5
conv2 = nn.Conv2d(1, 3, kernel_size=(5, 5), stride=1, padding=0, bias=False)
print(conv2(V(input)).size())

out = conv1(V(input))

plt.imshow(to_pil(out.squeeze(0)), cmap='gray')
plt.axis('off')
plt.show()
(3)池化层

池化层可看作是一种特殊的卷积层,用来下采样。但池化层没有可学习参数,其weight是固定的。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
from PIL import Image
import numpy as np
import matplotlib.pyplot as plt

import torch as t
import torch.nn as nn
from torch.autograd import Variable as V
from torchvision.transforms import ToTensor, ToPILImage

to_tensor = ToTensor() # img -> tensor
to_pil = ToPILImage() # tensor -> img

image = np.array(Image.open('lena.jpg'))

# 样本数=1,通道数=1,图像的shape是256乘256的
input = to_tensor(image).unsqueeze(0)
print(input.size())

# 池化层
pool = nn.AvgPool2d(2, 2) # 平均池化
pool = nn.MaxPool2d(2, 2) # 最大池化

out = pool(V(input))

plt.imshow(to_pil(out.squeeze(0)), cmap='gray')
plt.axis('off')
plt.show()
(4)RNN层与RNN单元

nn.RNN可以接收一个序列的输入

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import torch as t
import torch.nn as nn
from torch.autograd import Variable as V

# 构造一个输入序列, 6个批次, 每批3个序列样本, 特征维度是5
input = t.randn(6, 3, 5)
print(input)
print(input.size())

# 构造RNN网络, 输入维度5, 隐层的维度10, 网络的层数1
rnn = nn.RNN(5, 10, 1)

out, _ = rnn(V(input))

print(out)

nn.RNNCell只能接收序列中的单步的输入,且必须传入隐藏状态

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import torch as t
import torch.nn as nn
from torch.autograd import Variable as V

# 构造一个输入序列, 6个批次, 每批3个序列样本, 特征维度是5
input = t.randn(6, 3, 5)
# 隐层输入
input_h = t.randn(3, 10)

# 构造RNN单元, 输入维度5, 隐层的维度10
rnncell = nn.RNNCell(5, 10)

output = []
for i in range(input.size()[0]):
cell_out = rnncell(V(input[i]), input_h)
output.append(cell_out)

print("RNNCell的输出:\n", output)
(5)LSTM层与LSTM单元

nn.LSTM可以接收一个序列的输入,且必须传入隐藏状态

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import torch as t
import torch.nn as nn
from torch.autograd import Variable as V

# 构造一个输入序列, 6个批次, 每批3个序列样本, 特征维度是5
input = t.randn(6, 3, 5)
# 隐层输入, 网络层数1, 每批3个序列样本, 隐层的维度10
input_h0 = t.randn(1, 3, 10)
input_c0 = t.randn(1, 3, 10)

# 构造LSTM网络, 输入维度5, 隐层的维度10, 网络的层数1
lstm = nn.LSTM(5, 10, 1)

out, _ = lstm(V(input), (input_h0, input_c0))

print(out)

nn.LSTMCell只能接收序列中的单步的输入,且必须传入隐藏状态

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
import torch as t
import torch.nn as nn
from torch.autograd import Variable as V

# 构造一个输入序列, 6个批次, 每批3个序列样本, 特征维度是5
input = t.randn(6, 3, 5)
# 隐层输入
input_h0 = t.randn(3, 10)
input_c0 = t.randn(3, 10)

# 构造LSTM单元, 输入维度5, 隐层的维度10
lstmcell = nn.LSTMCell(5, 10)

output = []
for i in range(input.size()[0]):
cell_out = lstmcell(V(input[i]), (input_h0, input_c0))
output.append(cell_out)

print("LSTMCell的输出:\n", output)
(6)GRU层与GRU单元

nn.GRU可以接收一个序列的输入,,且必须传入隐藏状态

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import torch as t
import torch.nn as nn
from torch.autograd import Variable as V

# 构造一个输入序列, 6个批次, 每批3个序列样本, 特征维度是5
input = t.randn(6, 3, 5)
# 隐层输入, 网络层数1, 每批3个序列样本, 隐层的维度10
input_h = t.randn(1, 3, 10)

# 构造GRU网络, 输入维度5, 隐层的维度10, 网络的层数1
gru = nn.GRU(5, 10, 1)

out, _ = gru(V(input), input_h)

print(out)

nn.GRUCell只能接收序列中的单步的输入,且必须传入隐藏状态

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import torch as t
import torch.nn as nn
from torch.autograd import Variable as V

# 构造一个输入序列, 6个批次, 每批3个序列样本, 特征维度是5
input = t.randn(6, 3, 5)
# 隐层输入
input_h = t.randn(3, 10)

# 构造GRU单元, 输入维度5, 隐层的维度10
grucell = nn.GRUCell(5, 10)

output = []
for i in range(input.size()[0]):
cell_out = grucell(V(input[i]), input_h)
output.append(cell_out)

print("GRUCell的输出:\n", output)
(7)批规范化层,即归一化层
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
import torch as t
import torch.nn as nn
from torch.autograd import Variable as V

# 样本数为2, 维度为3
input = t.randn(2, 3)
print(input.size())

# 全连接层, 输入节点3, 输出节点4
linear = nn.Linear(3, 4)
linear_out = linear(input)
print(linear_out)

# 批规范化层
bn = nn.LayerNorm(4, eps=1e-05) # channel方向做归一化
# bn = nn.BatchNorm1d(4) # batch方向做归一化
# bn = nn.InstanceNorm2d(3) # 一个channel内做归一化(三维或更高维)
# bn = nn.GroupNorm(1, 4) # 将channel方向分group,然后每个group内做归一化,与batchsize无关
# bn = nn.LocalResponseNorm(2) # 与邻居通道做归一化(三维或更高维)

bn_out = bn(linear_out)
# 归一化后平均值为0
print(bn_out.mean(0))
# 归一化后方差为标准单位方差(由于计算无偏方差时分母会减1, 故使用unbiased=False确保分母不减1)
print(bn_out.var(0, unbiased=False))
(8)dropout正则化层
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import torch as t
import torch.nn as nn
from torch.autograd import Variable as V

# 样本数为2, 维度为3
input = t.randn(2, 3)

# 全连接层, 输入节点3, 输出节点4
linear = nn.Linear(3, 4)
linear_out = linear(input)

# 批规范化层
bn = nn.BatchNorm1d(4)

bn_out = bn(linear_out)

# dropout正则化层,每个元素以0.5的概率舍弃,实现dropout正则化,消除过拟合问题
dropout = nn.Dropout(0.5)
out = dropout(bn_out)
print(out)

二、激活函数

PyTorch实现了常见的激活函数,这些函数可作为独立的layer使用。

(1)Signmoid函数

1
2
3
4
5
6
7
8
9
10
11
12
13
import torch as t
import torch.nn as nn
from torch.autograd import Variable as V

# 输入数据
input = t.randn(5)
print(input)

# 激活函数
activation = nn.Sigmoid()

out = activation(V(input))
print(out)

(2)Tanh函数

1
2
3
4
5
6
7
8
9
10
11
12
13
import torch as t
import torch.nn as nn
from torch.autograd import Variable as V

# 输入数据
input = t.randn(5)
print(input)

# 激活函数
activation = nn.Tanh()

out = activation(V(input))
print(out)

(3)Relu函数

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import torch as t
import torch.nn as nn
from torch.autograd import Variable as V

# 输入数据
input = t.randn(5)
print(input)

# 激活函数
activation = nn.ReLU()

# 小于0的变成0
out = activation(V(input))
print(out)

(4)LeakyReLU函数

1
2
3
4
5
6
7
8
9
10
11
12
13
import torch as t
import torch.nn as nn
from torch.autograd import Variable as V

# 输入数据
input = t.randn(5)
print(input)

# 激活函数
activation = nn.LeakyReLU(0.1)

out = activation(V(input))
print(out)

其他激活函数

三、神经网络建模

构建自己的网络就是定义nn.Module的一个子类,这时需要重写初始化函数init()和前向过程forward()。init()函数中需要调用父类的初始化函数,声明神经网络的层结构;forward()函数用于构建网络从输入到输出的过程。

1、序贯建模方法

nn.Sequential是一个Sequential容器,模型的各层被顺序添加到容器中。缺点是每层的编号是默认的阿拉伯数字,不易区分。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import torch.nn as nn
from collections import OrderedDict

class MyNet(nn.Module):
def __init__(self):
super(MyNet, self).__init__()
self.conv = nn.Sequential(
nn.Conv2d(3, 32, 3, 1, 1),
nn.ReLU(),
nn.MaxPool2d(2))
self.dense = nn.Sequential(
# 通过字典的形式添加每一层,并且设置单独的层名称
OrderedDict([
("Linear1", nn.Linear(32 * 3 * 3, 128)),
("ReLU", nn.ReLU()),
("Linear2", nn.Linear(128, 10))
])
)

def forward(self, x):
x = self.conv(x)
x = self.dense(x)
return x

print("MyNet:")
model = MyNet()
print(model)

也可以通过add_module()为Sequential容器添加每一层,并且为每一层增加了一个单独的名字。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import torch.nn as nn

class MyNet(nn.Module):
def __init__(self):
super(MyNet, self).__init__()
self.conv = nn.Sequential()
self.conv.add_module("conv1",nn.Conv2d(3, 32, 3, 1, 1))
self.conv.add_module("relu1",nn.ReLU())
self.conv.add_module("pool1",nn.MaxPool2d(2))
self.dense = nn.Sequential()
self.dense.add_module("dense1",nn.Linear(32 * 3 * 3, 128))
self.dense.add_module("relu2",nn.ReLU())
self.dense.add_module("dense2",nn.Linear(128, 10))

def forward(self, x):
x = self.conv(x)
x = self.dense(x)
return x

print("MyNet:")
model = MyNet()
print(model)

2、函数式建模方法

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import torch.nn as nn

class MyNet(nn.Module):
def __init__(self):
super(MyNet, self).__init__() # 调用父类的初始化函数
self.layer1 = nn.Linear(100, 200)
self.layer2 = nn.Linear(200, 10)

def forward(self, x):
x = self.layer1(x)
x = nn.relu(x)
x = self.layer2(x)
x = nn.softmax(x)
return x

print("MyNet:")
model = MyNet()
print(model)
0%