在不同数据维度上评估模型
Posted
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了在不同数据维度上评估模型相关的知识,希望对你有一定的参考价值。
[我具有五种形状(37520, 32, 9)
的形状(37520, 5)
的数据,我正在使用Conv1D训练模型,到目前为止,我已经能够训练数据。但是问题是我需要在不同的维度上进行评估-(37520, 32, 4)
(类相同),出现以下错误:
Traceback (most recent call last):
File "data_maker_cnn_multiuser_folds_correct.py", line 870, in <module>
f = run_cnn(a)
File "data_maker_cnn_multiuser_folds_correct.py", line 155, in run_cnn
_, accuracy = model.evaluate(x_test, y_test, batch_size=batch_size, verbose=verbose)
File "/Users/akshayrajgollahalli/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py", line 930, in evaluate
use_multiprocessing=use_multiprocessing)
File "/Users/akshayrajgollahalli/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 490, in evaluate
use_multiprocessing=use_multiprocessing, **kwargs)
File "/Users/akshayrajgollahalli/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 426, in _model_iteration
use_multiprocessing=use_multiprocessing)
File "/Users/akshayrajgollahalli/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 646, in _process_inputs
x, y, sample_weight=sample_weights)
File "/Users/akshayrajgollahalli/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py", line 2383, in _standardize_user_data
batch_size=batch_size)
File "/Users/akshayrajgollahalli/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py", line 2410, in _standardize_tensors
exception_prefix='input')
File "/Users/akshayrajgollahalli/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_utils.py", line 582, in standardize_input_data
str(data_shape))
ValueError: Error when checking input: expected conv1d_input to have shape (32, 9) but got array with shape (32, 4)
使用以下代码:
def run_cnn(data_dict: Dict[str, np.ndarray]):
"""
Runs a 1D CNN.
:param data_dict: A dictionary of training, testing and their targets (Ndarray).
:return:
"""
x_train = data_dict['training_data']
x_test = data_dict['testing_data']
y_train = np.expand_dims(data_dict['training_target'], axis=1)
y_test = np.expand_dims(data_dict['testing_target'], axis=1)
y_train = y_train - 1
y_test = y_test - 1
y_train = tf.keras.utils.to_categorical(y_train)
y_test = tf.keras.utils.to_categorical(y_test)
print(y_test.shape)
print(x_train.shape, y_train.shape, x_test.shape, y_test.shape)
verbose = 1
epochs = 2
batch_size = 32
n_timesteps, n_features, n_outputs = x_train.shape[1], x_train.shape[2], y_train.shape[1]
print("Number of time steps: ", n_features)
print("Number of features: ", n_features)
print("Number of outputs: ", n_outputs)
model = tf.keras.Sequential()
model.add(
tf.keras.layers.Conv1D(filters=64, kernel_size=3, activation='relu',
input_shape=(n_timesteps, n_features)))
model.add(tf.keras.layers.Conv1D(filters=64, kernel_size=3, activation='relu'))
model.add(tf.keras.layers.Dropout(0.5))
model.add(tf.keras.layers.MaxPooling1D(pool_size=2))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(100, activation='relu'))
model.add(tf.keras.layers.Dense(n_outputs, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit network
history = model.fit(x_train, y_train, epochs=epochs, batch_size=batch_size, verbose=verbose)
# evaluate model
_, accuracy = model.evaluate(x_test, y_test, batch_size=batch_size, verbose=verbose)
return accuracy
甚至有可能这样做吗?我也尝试使用预测,但是仍然出现错误。
答案
模型需要固定以下参数才能进行训练和评估。
- 模型中所有图层的名称和类型。
- 输出每一层的形状。
- 每层的权重参数数。
- 每一层接收的输入。
- 模型的可训练和不可训练参数的总数。
因此,对于在另一个输入形状上训练过的模型,您将无法在不同的输入形状上进行评估。
您可以像@HitLuca一样进行人为修复,但结果可能不佳。
另一个选项是将模型的原始输入大小截断为(32,4)
并重新训练。虽然会丢失信息。训练后,您的评估将起作用。但是,如果您尝试使用原始输入大小(32,9)
进行评估,它将无法正常工作。
希望这能回答您的问题。祝您学习愉快。
以上是关于在不同数据维度上评估模型的主要内容,如果未能解决你的问题,请参考以下文章