如何在张量流中创建独立的 LSTM 单元？

Posted 2023-02-16

【中文标题】如何在张量流中创建独立的 LSTM 单元？

【英文标题】：How to create independent LSTM cells in tensorflow?

【发布时间】：2018-04-19 10:05:45

【问题描述】：

我正在尝试制作一个 RNN 分类器，它有 3 个不同的时间序列，每个时间序列有 3 个维度作为输入，并且时间序列可以有不同的长度。所以为了解决这个问题，我对 3 个 RNN 建模并在最后一层将它们连接起来。

但是，我收到以下错误消息：

ValueError: 变量 rnn/multi_rnn_cell/cell_0/basic_lstm_cell/kernel 已经存在，不允许。你的意思是设置reuse=True 变量范围？

timeSeries = ['outbound', 'rest', 'return']
n_steps = 
    'outbound': 3159,
    'rest': 3603,
    'return': 3226

n_inputs = 3
n_neurons = 20
n_outputs = 2
n_layers = 1

learning_rate = 0.001


y = tf.placeholder(tf.int32, [None], name="y")
X = 
seq_length = 
for timeSeriesName in timeSeries:
    with tf.name_scope(timeSeriesName + "_placeholders") as scope:
        X[timeSeriesName] = tf.placeholder(tf.float32, [None, n_steps[timeSeriesName], n_inputs])
        seq_length[timeSeriesName] = tf.placeholder(tf.int32, [None])


outputs = 
states = 
top_layer_h_state = 
lstm_cells = 
multi_cell = 
finalRNNlayers = []
for timeSeriesName in timeSeries:
    with tf.name_scope(timeSeriesName) as scope:
        lstm_cells[timeSeriesName] = [tf.contrib.rnn.BasicLSTMCell(num_units=n_neurons)
                                      for layer in range(n_layers)]
        multi_cell[timeSeriesName] = tf.contrib.rnn.MultiRNNCell(lstm_cells[timeSeriesName])
        outputs[timeSeriesName], states[timeSeriesName] = tf.nn.dynamic_rnn(
            multi_cell[timeSeriesName], X[timeSeriesName], dtype=tf.float32,
            sequence_length=seq_length[timeSeriesName])
        top_layer_h_state[timeSeriesName] = states[timeSeriesName][-1][1]
        finalRNNlayers.append(top_layer_h_state[timeSeriesName])

with tf.name_scope("3Stages_mixed") as scope:
    concat3_top_layer_h_states = tf.concat(finalRNNlayers, axis=1)
    logits = tf.layers.dense(concat3_top_layer_h_states, n_outputs, name="softmax")

我希望每个时间序列都有独立的 LSTM 单元，每个单元都有自己的权重，所以不能重用，这个错误应该如何解决？

The full traceback of the error can be found here.

【参考方案1】：

将 tf.name_scope(timeSeriesName) 更改为 tf.variable_scope(timeSeriesName)。 tf.name_scope 和 tf.variable_scope 之间的区别在 this quesion 中讨论。在您的情况下，重要的是 tf.get_variable 忽略名称范围，并且 LSTM 单元参数完全使用 tf.get_variable 创建。

示例代码查看区别：

import tensorflow as tf

state = tf.zeros([32, 6])

input1 = tf.placeholder(tf.float32, [32, 10])
input2 = tf.placeholder(tf.float32, [32, 10])

# Works ok:
with tf.variable_scope('scope-1'):
  tf.nn.rnn_cell.BasicLSTMCell(3, state_is_tuple=False)(input1, state)
with tf.variable_scope('scope-2'):
  tf.nn.rnn_cell.BasicLSTMCell(3, state_is_tuple=False)(input2, state)

# Fails:
with tf.name_scope('name-1'):
  tf.nn.rnn_cell.BasicLSTMCell(3, state_is_tuple=False)(input1, state)
with tf.name_scope('name-2'):
  tf.nn.rnn_cell.BasicLSTMCell(3, state_is_tuple=False)(input2, state)