简单的 TensorFlow LSTM 网络：ValueError：没有为任何变量提供梯度

Posted 2023-02-23

【中文标题】简单的 TensorFlow LSTM 网络：ValueError：没有为任何变量提供梯度

【英文标题】：Simple TensorFlow LSTM Network: ValueError: No gradients provided for any variable

【发布时间】：2021-02-27 19:00:43

【问题描述】：

我正在尝试创建一个简单、简单的 LSTM 示例来预测浮点数序列中的下一个点。为了简单起见，我选择了线性样本数据。我正在使用 TensorFlow 2.3.1 和 NumPy 1.18.5。

这是输入数据的设置，长度为 5 的基本滚动范围：

# Sample data -- linear function
linear_data=0.05+0.05*np.array(range(20))

# Size of window in rolling horizon
window = 5

# Tensorflow expects 3D: i,j,k = samples, window/sequence data, 
# features (one since we're univariate)
rolling_x = np.zeros([len(linear_data)-window, window, 1])
rolling_y = np.zeros([len(linear_data)-window, 1])

# Populate the rolling horizon data
for idx in [t for t in range(len(linear_data)-window)]:
    rolling_x[idx, :, 0] = linear_data[idx:idx+window]
    rolling_y[idx, :] = linear_data[idx+window]
    
print(rolling_x[:,:,0])

>>> [[0.05 0.1  0.15 0.2  0.25]
>>> [0.1  0.15 0.2  0.25 0.3 ]
>>> [0.15 0.2  0.25 0.3  0.35]
...
>>> [0.75 0.8  0.85 0.9  0.95]]

其中rolling_y 包含序列中的下一项。我的理解是，数据应该由第一个轴上的批次/样本数据、第二个轴上的序列/窗口和最后一个轴上的特征数（在这种情况下为 1，因为我是单变量的）来构建。

从这里我构建了一个非常简单的模型：

# Build the model
tf_model = Sequential()
tf_model.add(LSTM(
    units=32,
    input_shape=[window, 1]
))
tf_model.add(Dense(units=1))
tf_model.compile()

它编译得很好，但是当我尝试训练 (tf_model.fit(rolling_x, rolling_y)) 时出现以下错误：

    c:\users\____\documents\data_analytics\lstm-demo\env\lib\site-packages\tensorflow\python\keras\engine\training.py:806 train_function  *
        return step_function(self, iterator)
    c:\users\____\documents\data_analytics\lstm-demo\env\lib\site-packages\tensorflow\python\keras\engine\training.py:796 step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    c:\users\____\documents\data_analytics\lstm-demo\env\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:1211 run
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    c:\users\____\documents\data_analytics\lstm-demo\env\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:2585 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    c:\users\____\documents\data_analytics\lstm-demo\env\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:2945 _call_for_each_replica
        return fn(*args, **kwargs)
    c:\users\____\documents\data_analytics\lstm-demo\env\lib\site-packages\tensorflow\python\keras\engine\training.py:789 run_step  **
        outputs = model.train_step(data)
    c:\users\____\documents\data_analytics\lstm-demo\env\lib\site-packages\tensorflow\python\keras\engine\training.py:756 train_step
        _minimize(self.distribute_strategy, tape, self.optimizer, loss,
    c:\users\____\documents\data_analytics\lstm-demo\env\lib\site-packages\tensorflow\python\keras\engine\training.py:2736 _minimize
        gradients = optimizer._aggregate_gradients(zip(gradients,  # pylint: disable=protected-access
    c:\users\____\documents\data_analytics\lstm-demo\env\lib\site-packages\tensorflow\python\keras\optimizer_v2\optimizer_v2.py:562 _aggregate_gradients
        filtered_grads_and_vars = _filter_grads(grads_and_vars)
    c:\users\____\documents\data_analytics\lstm-demo\env\lib\site-packages\tensorflow\python\keras\optimizer_v2\optimizer_v2.py:1270 _filter_grads
        raise ValueError("No gradients provided for any variable: %s." %

    ValueError: No gradients provided for any variable: ['lstm_3/lstm_cell_3/kernel:0', 'lstm_3/lstm_cell_3/recurrent_kernel:0', 'lstm_3/lstm_cell_3/bias:0', 'dense_3/kernel:0', 'dense_3/bias:0'].

【问题讨论】：

您没有指定训练损失。老实说，我很惊讶没有为此采取任何保障措施。我建议再看看compile 和它需要的参数。

我这么简单的疏忽，确实解决了问题。我会考虑添加一个功能请求，因为我认为它会默认为 RMSE、MSE 或 MAE。它可能对错误消息更有帮助。

【参考方案1】：

正如@xdurch0 所说，我们必须在编译模型时指定训练配置优化器、损失和指标。

请参考下图compile参数进行训练。

compile(
    optimizer='rmsprop', loss=None, metrics=None, loss_weights=None,
    weighted_metrics=None, run_eagerly=None, **kwargs
)

为了社区的利益，我在这里发布完整的工作代码

import tensorflow as tf
import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, LSTM

# Sample data -- linear function
linear_data=0.05+0.05*np.array(range(20))

# Size of window in rolling horizon
window = 5

# Tensorflow expects 3D: i,j,k = samples, window/sequence data, 
# features (one since we're univariate)
rolling_x = np.zeros([len(linear_data)-window, window, 1])
rolling_y = np.zeros([len(linear_data)-window, 1])

# Populate the rolling horizon data
for idx in [t for t in range(len(linear_data)-window)]:
    rolling_x[idx, :, 0] = linear_data[idx:idx+window]
    rolling_y[idx, :] = linear_data[idx+window]
    
print(rolling_x[:,:,0])

# Build the model
tf_model = Sequential()
tf_model.add(LSTM(
    units=32,
    input_shape=[window, 1]
))
tf_model.add(Dense(units=1))
tf_model.compile(optimizer='adam', loss='mae', metrics=['mae'])
tf_model.fit(rolling_x, rolling_y)

输出：

[[0.05 0.1  0.15 0.2  0.25]
 [0.1  0.15 0.2  0.25 0.3 ]
 [0.15 0.2  0.25 0.3  0.35]
 [0.2  0.25 0.3  0.35 0.4 ]
 [0.25 0.3  0.35 0.4  0.45]
 [0.3  0.35 0.4  0.45 0.5 ]
 [0.35 0.4  0.45 0.5  0.55]
 [0.4  0.45 0.5  0.55 0.6 ]
 [0.45 0.5  0.55 0.6  0.65]
 [0.5  0.55 0.6  0.65 0.7 ]
 [0.55 0.6  0.65 0.7  0.75]
 [0.6  0.65 0.7  0.75 0.8 ]
 [0.65 0.7  0.75 0.8  0.85]
 [0.7  0.75 0.8  0.85 0.9 ]
 [0.75 0.8  0.85 0.9  0.95]]

1/1 [==============================] - 0s 2ms/step - loss: 0.6889 - mae: 0.6889
<tensorflow.python.keras.callbacks.History at 0x7fa76e2b05f8>