RNN流程理解

Posted 2022-02-05 三つ叶

import torch.nn as nn
import torch

rnn = nn.RNN(10, 20, 2)  # 约定了此 RNN 的输入 input_feature_size=10, output_feature_size=20, num_layers=2
print(rnn)

input = torch.randn(5, 3, 10)  # input=(input_sequence, batch_size, input_feature_size)

h0 = torch.randn(2, 3, 20)  # h0=(num_layers, batch_size, output_feature_size)

output, hn = rnn(input, h0)
print(output.shape)  # output=(input_sequence, batch_size, output_feature_size)
print(hn.shape)   # hn=(num_layers, batch_size, output_feature_size)

以下流程对照参考这一节内容

$$\begin{gathered} X--->(Sequence\_length,batch\_size,input\_feature\_size) \\ {X}_1,X_2\dots \dots X_{sequence\_length}--->(batch\_size,input\_feature\_size) \\ {W}_{xh}-->(input\_feature\_size,hidden\_layers) \\ {W}_{hh}-->(hidden\_layers,hidden\_layers) \\ {W}_{hq}-->(hidden\_layers,output\_feature\_size) \end{gathered}$$

归根到底，RNN 其实仍旧是神经网络，不要想得太复杂

在 Pytorch 中 hidden_layers 和 output_feature_size 是相同的