使用 dropout (TF2.0) 时，可变批量大小不适用于 tf.keras.layers.RNN？

Posted 2023-02-16

【中文标题】使用 dropout (TF2.0) 时，可变批量大小不适用于 tf.keras.layers.RNN？

【英文标题】：Variable batch sizes don't work with tf.keras.layers.RNN when dropout is used (TF2.0)?

【发布时间】：2020-02-26 06:29:38

【问题描述】：

我想将 RNN 包装器与多个 LSTM 单元一起使用，并带有 dropout。但是，如果批量大小发生变化，我会收到错误消息。

当我删除 dropout 时，代码可以正常工作，所以我认为问题是批次之间没有重置 dropout 掩码。

import numpy as np
import tensorflow as tf

input_dim = 3
output_dim = 3
num_timesteps = 2
neurons = [32,32]

# Model
input_layer = tf.keras.Input(shape=(num_timesteps, input_dim))
cell = [tf.keras.layers.LSTMCell(n,dropout=.2) for n in neurons]
lstm = tf.keras.layers.RNN(cell,return_state=True,return_sequences=True)
lstm_out, hidden_state, cell_state = lstm(input_layer)
output = tf.keras.layers.Dense(output_dim)(lstm_out)

mdl = tf.keras.Model(
    inputs=input_layer,
    outputs=[hidden_state, cell_state, output]
)

# Run batches of different sizes
batch_1 = np.random.rand(10, num_timesteps, input_dim).astype(np.float32)
h_state, c_state, out = mdl(batch_1) # batch size is 10x2x3

batch_2 = np.random.rand(9, num_timesteps, input_dim).astype(np.float32)
h_state, c_state, out = mdl(batch_2) # batch size is 9x2x3

此代码给出错误：InvalidArgumentError: Incompatible shapes: [9,3] vs. [10,3] [Op:Mul] name: model/rnn/mul/

如果我去掉 dropout，代码就可以工作。我可以以某种方式使用 reset_dropout_mask 吗？它似乎没有被调用。

【参考方案1】：

我可以使用Tensorflow Version 2.0.0 重现您的错误。

但是，如果我将 Tensorflow Version 升级到 2.1 或 2.2 并运行相同的代码，则不会出现错误。

完整的工作代码如下所示：

!pip install tensorflow==2.2

import numpy as np
import tensorflow as tf

print(tf.__version__) # Printing the Tensorflow Version just to be Sure

input_dim = 3
output_dim = 3
num_timesteps = 2
neurons = [32,32]

# Model
input_layer = tf.keras.Input(shape=(num_timesteps, input_dim))
cell = [tf.keras.layers.LSTMCell(n,dropout=.2) for n in neurons]
lstm = tf.keras.layers.RNN(cell,return_state=True,return_sequences=True)
lstm_out, hidden_state, cell_state = lstm(input_layer)
output = tf.keras.layers.Dense(output_dim)(lstm_out)

mdl = tf.keras.Model(
    inputs=input_layer,
    outputs=[hidden_state, cell_state, output]
)

# Run batches of different sizes
batch_1 = np.random.rand(10, num_timesteps, input_dim).astype(np.float32)
h_state, c_state, out = mdl(batch_1) # batch size is 10x2x3

batch_2 = np.random.rand(9, num_timesteps, input_dim).astype(np.float32)
h_state, c_state, out = mdl(batch_2) # batch size is 9x2x3

以上代码的输出如下所示：

2.2.0

希望这会有所帮助。快乐学习！