Tensorflow中循环神经网络及其Wrappers

Posted 冬色

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Tensorflow中循环神经网络及其Wrappers相关的知识,希望对你有一定的参考价值。

  • tf.nn.rnn_cell.LSTMCell

    • 又名:tf.nn.rnn_cell.BasicLSTMCelltf.contrib.rnn.LSTMCell

    • 参见: tf.nn.rnn_cell.LSTMCell

    • 输出:

      • output:LSTM单元输出,与LSTM cell state的区别在于该输出又经过激活以及和一个sigmoid函数输出相乘。shape: [batch_size,num_units]
      • new_state:当前时间步上的LSTM cell stateLSTM output。使用数据结构LSTMStateTuple描述,LSTMStateTuple:(c,h),其中,h与上述的output完全相同。shape: ([batch_size,num_units],[batch_size,num_units])
    • 示例:

      batch_size=10
      embedding_size=300
      inputs=tf.Variable(tf.random_normal([batch_size,embedding_size]))
      previous_state=(tf.Variable(tf.random_normal([batch_size,128])),tf.Variable(tf.random_normal([batch_size,128])))
      lstmcell=tf.nn.rnn_cell.LSTMCell(128)
      outputs,states=lstmcell(inputs,previous_state)

      输出:

      outputs:
      <tf.Tensor ‘lstm_cell/mul_2:0‘ shape=(10, 128) dtype=float32>
      
      states:
      LSTMStateTuple(c=<tf.Tensor ‘lstm_cell/add_1:0‘ shape=(10, 128) dtype=float32>, h=<tf.Tensor ‘lstm_cell/mul_2:0‘ shape=(10, 128) dtype=float32>)
  • tf.nn.rnn_cell.MultiRNNCell

    • 参见:tf.nn.rnn_cell.MultiRNNCell

    • 输出:

      • outputs: 最顶层cell的最后一个时间步的输出。shape:[batch_size,cell.output_size]
      • states:每一层的state,M层LSTM则输出M个LSTMStateTuple组成的Tuple。
    • 示例:

      batch_size=10
      inputs=tf.Variable(tf.random_normal([batch_size,128]))
      previous_state0=(tf.random_normal([batch_size,100]),tf.random_normal([batch_size,100]))
      previous_state1=(tf.random_normal([batch_size,200]),tf.random_normal([batch_size,200]))
      previous_state2=(tf.random_normal([batch_size,300]),tf.random_normal([batch_size,300]))
      num_units=[100,200,300]
      cells=[tf.nn.rnn_cell.LSTMCell(num_unit) for num_unit in num_units]
      mul_cells=tf.nn.rnn_cell.MultiRNNCell(cells)
      outputs,states=mul_cells(inputs,(previous_state0,previous_state1,previous_state2))

      输出:

      outputs:
      <tf.Tensor ‘multi_rnn_cell_1/cell_2/lstm_cell/mul_2:0‘ shape=(10, 300) dtype=float32>
      
      states:
      (LSTMStateTuple(c=<tf.Tensor ‘multi_rnn_cell_1/cell_0/lstm_cell/add_1:0‘ shape=(10, 100) dtype=float32>, h=<tf.Tensor ‘multi_rnn_cell_1/cell_0/lstm_cell/mul_2:0‘ shape=(10, 100) dtype=float32>),
       LSTMStateTuple(c=<tf.Tensor ‘multi_rnn_cell_1/cell_1/lstm_cell/add_1:0‘ shape=(10, 200) dtype=float32>, h=<tf.Tensor ‘multi_rnn_cell_1/cell_1/lstm_cell/mul_2:0‘ shape=(10, 200) dtype=float32>),
       LSTMStateTuple(c=<tf.Tensor ‘multi_rnn_cell_1/cell_2/lstm_cell/add_1:0‘ shape=(10, 300) dtype=float32>, h=<tf.Tensor ‘multi_rnn_cell_1/cell_2/lstm_cell/mul_2:0‘ shape=(10, 300) dtype=float32>))
  • tf.nn.dynamic_rnn

    • 参见:tf.nn.dynamic_rnn

    • 输出:

      • outputs: 每个时间步上的LSTM输出;若有多层LSTM,则为每一个时间步上最顶层的LSTM的输出。shape: [batch_size,max_time,cell.output_size]
      • state:最后一个时间步的状态,该状态使用LSTMStateTuple结构输出;若有M层LSTM,则输出M个LSTMStateTuple。shape:单层LSTM输出:[batch_size,cell.output_size];M层LSTM输出:M个LSTMStateTuple组成的Tuple,这也即是说:outputs[:,-1,:]==state[-1,:,:]。
    • 示例:

      batch_size=10
      max_time=20
      data=tf.Variable(tf.random_normal([batch_size,max_time,128]))
      # create a BasicRNNCell
      rnn_cell = tf.nn.rnn_cell.BasicRNNCell(num_units=128)
      
      # defining initial state
      initial_state = rnn_cell.zero_state(batch_size,dtype=tf.float32)
      
      # ‘outputs‘ is a tensor of shape [batch_size, max_time, cell_state_size]
      # ‘state‘ is a tensor of shape [batch_size, cell_state_size]
      outputs, state = tf.nn.dynamic_rnn(cell=rnn_cell, inputs=data,
                                         initial_state=initial_state,
                                         dtype=tf.float32)

      输出:

      outpus:
      <tf.Tensor ‘rnn_2/transpose_1:0‘ shape=(10, 20, 128) dtype=float32>
      
      state:
      <tf.Tensor ‘rnn_2/while/Exit_3:0‘ shape=(10, 128) dtype=float32>
      batch_size=10
      max_time=20
      data=tf.Variable(tf.random_normal([batch_size,max_time,128]))
      # create 2 LSTMCells
      rnn_layers = [tf.nn.rnn_cell.LSTMCell(size) for size in [128, 256]]
      
      # create a RNN cell composed sequentially of a number of RNNCells
      multi_rnn_cell = tf.nn.rnn_cell.MultiRNNCell(rnn_layers)
      
      # ‘outputs‘ is a tensor of shape [batch_size, max_time, 256]
      # ‘state‘ is a N-tuple where N is the number of LSTMCells containing a
      # tf.contrib.rnn.LSTMStateTuple for each cell
      outputs, state = tf.nn.dynamic_rnn(cell=multi_rnn_cell,
                                         inputs=data,
                                         dtype=tf.float32)
      outputs:
      <tf.Tensor ‘rnn_1/transpose_1:0‘ shape=(10, 20, 256) dtype=float32>
      
      state:
      (LSTMStateTuple(c=<tf.Tensor ‘rnn_1/while/Exit_3:0‘ shape=(10, 128) dtype=float32>, h=<tf.Tensor ‘rnn_1/while/Exit_4:0‘ shape=(10, 128) dtype=float32>),
       LSTMStateTuple(c=<tf.Tensor ‘rnn_1/while/Exit_5:0‘ shape=(10, 256) dtype=float32>, h=<tf.Tensor ‘rnn_1/while/Exit_6:0‘ shape=(10, 256) dtype=float32>))
  • tf.nn.bidirectional_dynamic_rnn

    • 参见:tf.nn.bidirectional_dynamic_rnn

    • 输出:

      • outputs:(output_fw,output_bw):前向cell+后向cell

        其中,output_fw、output_bw均为:[batch_size,max_time,cell.output_size]

      • state:(output_state_fw,output_state_bw):包含前向和后向隐状态组成的元组

        其中,output_state_fw、output_state_bw均为LSTMStateTuple。LSTMStateTuple:(c,h),分别为cell_state,hidden_output

  • tf.contrib.seq2seq.dynamic_decode

    • 输出:
      • final_outputs,包含rnn_output和sample_id,分别可用final_output.rnn_output和final_outputs.sample_id获取到。
      • final_state,可以从最后一个解码器状态获取alignments,alignments = tf.transpose(final_decoder_state.alignment_history.stack(), [1, 2, 0])
      • final_sequence_lengths

以上是关于Tensorflow中循环神经网络及其Wrappers的主要内容,如果未能解决你的问题,请参考以下文章

TensorFlow从0到1之TensorFlow超参数及其调整(24)

循环神经网络系列Tensorflow中dynamic_rnn

TensorFlow:简单的循环神经网络

tensorflow 基础学习十:RNN

机器学习与Tensorflow——循环神经网络长短时记忆网络

TensorFlow框架之RNN循环神经网络详解