Keras DNN 预测模型准确率没有提高

Posted 2023-02-16

【中文标题】Keras DNN 预测模型准确率没有提高

【英文标题】：Keras DNN prediction model Accuracy is not improving

【发布时间】：2020-11-20 23:35:54

【问题描述】：

我正在尝试使用 LUT 数据训练 Keras DNN 模型进行预测。我已经对数据进行了规范化并分为训练、测试和验证部分。我的晒黑和验证准确性（几乎）保持不变时遇到了问题。准确率始终停留在 (0.1431)。

我尝试了许多不同的超参数，包括将激活函数更改为 tanh 和 relu，我尝试在第一个密集层之后添加批量归一化层，我使用了 SGD 优化器（更改了学习率、动量，甚至尝试将优化器更改为 Adam），尝试不同的损失函数，添加/删除 dropout 层。

import pandas as pd
import numpy as np
import tensorflow as tf
from sklearn import preprocessing
from sklearn.metrics import explained_variance_score, \
     mean_absolute_error, \
     median_absolute_error
from sklearn.model_selection import train_test_split
##########################################################
# for DNN model
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
from tensorflow import feature_column
import os
import datetime
from sklearn.preprocessing import StandardScaler,MinMaxScaler

df=pd.read_csv("..../LUT.csv")

标准化数据（0到1之间）

scaler = MinMaxScaler()
df[df.columns] = scaler.fit_transform(df[df.columns].to_numpy())
# X will be a pandas dataframe of all columns except meantempm
X = df[[col for col in df.columns if col != 'TT']]
# y will be a pandas series of the meantempm
Y = df['TT']

使用 sklearn.model_selection.traing_test_split 将数据拆分为训练集和临时集

X_train, X_tmp, y_train, y_tmp = train_test_split(X, Y,  test_size=0.20, random_state=23)
# take the remaining 20% of data in X_tmp, y_tmp and split them evenly
X_test, X_val, y_test, y_val = train_test_split(X_tmp, y_tmp, test_size=0.5, random_state=23)

X_train.shape, X_test.shape, X_val.shape
print("Training instances   , Training features   ".format(X_train.shape[0], X_train.shape[1]))
print("Validation instances , Validation features ".format(X_val.shape[0], X_val.shape[1]))
print("Testing instances    , Testing features    ".format(X_test.shape[0], X_test.shape[1]))

使用 TensorFlow 编码从数组中创建 Keras 密集特征层。我们将在 Keras 模型构建期间使用该层来定义模型训练特征：

feature_columns = [feature_column.numeric_column(x) for x in X.columns]
feature_layer = tf.keras.layers.DenseFeatures(feature_columns)
feature_layer

创建张量流格式数据集的功能

def df_to_dataset(x,y, shuffle=True, batch_size=32):
    dataframe = x.copy()
    labels = y.copy()
    ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))
    if shuffle:
      ds = ds.shuffle(buffer_size=len(dataframe))
    ds = ds.batch(batch_size)
   return ds

接下来，借助实用函数将 Pandas 数据帧转换为 tf.data：

  batch_size = 250
  train_ds = df_to_dataset(X_train,y_train, batch_size=batch_size)
  val_ds = df_to_dataset(X_val,y_val, shuffle=False, batch_size=batch_size)
  test_ds = df_to_dataset(X_test,y_test, shuffle=False, batch_size=batch_size)

型号：

  #relu,sigmoid,tanh
  def get_compiled_model():
     model = keras.Sequential([
         feature_layer,
         layers.Dense(50, activation="tanh"),
         tf.keras.layers.Dropout(0.1),
         layers.Dense(35, activation='tanh'),
         layers.Dense(20, activation='tanh'),
         # layers.Dense(100, activation='tanh'),
         # tf.keras.layers.Dropout(0.1),
         layers.Dense(1,activation="linear")
            ])

  # Compile the model with the specified loss function.
   model.compile(optimizer=keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08),
            loss='mse',
            metrics=["accuracy",'mape',"RootMeanSquaredError"])
   return model

训练模型：

  # Callbacks time
  logdir = os.path.join("logs", datetime.datetime.now().strftime("%Y%m%d-%H%M%S"))
  tensorboard_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
   es = EarlyStopping(monitor='val_loss', patience=10)
   mcp = ModelCheckpoint(filepath='best_model_GPU_V1.h5', monitor='val_loss', save_best_only=True)

  # Create a MirroredStrategy.
   strategy = tf.distribute.MirroredStrategy()
   print("Number of devices: ".format(strategy.num_replicas_in_sync))

  # Open a strategy scope.
  with strategy.scope():
   # Everything that creates variables should be under the strategy scope.
   # In general this is only model construction & `compile()`.
    model = get_compiled_model()

  # Train the model on all available devices.
   EPOCHS = 50
   history = model.fit(train_ds,
          epochs=EPOCHS,
          # steps_per_epoch=1000,
          callbacks=[tensorboard_callback,es,mcp],
          validation_data=val_ds
          )

培训结果：

 Epoch 40/50
 621/621 [==============================] - 4s 7ms/step - root_mean_squared_error: 0.0202 - loss: 
 4.0961e-04 - mape: 1093214.5000 - accuracy: 0.1431 - val_root_mean_squared_error: 0.0124 - val_loss: 
 1.5268e-04 - val_mape: 509855.8438 - val_accuracy: 0.1464
 Epoch 41/50
 621/621 [==============================] - 4s 6ms/step - root_mean_squared_error: 0.0201 - loss: 
 4.0516e-04 - mape: 1089531.5000 - accuracy: 0.1431 - val_root_mean_squared_error: 0.0115 - val_loss: 
 1.3204e-04 - val_mape: 527368.5000 - val_accuracy: 0.1464
 Epoch 42/50
 621/621 [==============================] - 4s 7ms/step - root_mean_squared_error: 0.0199 - loss: 
 3.9764e-04 - mape: 1048669.6250 - accuracy: 0.1431 - val_root_mean_squared_error: 0.0107 - val_loss: 
 1.1494e-04 - val_mape: 543746.5625 - val_accuracy: 0.1464
 Epoch 43/50
 621/621 [==============================] - 4s 7ms/step - root_mean_squared_error: 0.0198 - loss: 
 3.9081e-04 - mape: 1053232.5000 - accuracy: 0.1431 - val_root_mean_squared_error: 0.0111 - val_loss: 
 1.2281e-04 - val_mape: 659315.5000 - val_accuracy: 0.1464
 Epoch 44/50
 621/621 [==============================] - 4s 7ms/step - root_mean_squared_error: 0.0196 - loss: 
 3.8481e-04 - mape: 1046033.1250 - accuracy: 0.1431 - val_root_mean_squared_error: 0.0132 - val_loss: 
 1.7504e-04 - val_mape: 944899.8125 - val_accuracy: 0.1464
 Epoch 45/50
 621/621 [==============================] - 4s 7ms/step - root_mean_squared_error: 0.0196 - loss: 
 3.8521e-04 - mape: 1033596.6875 - accuracy: 0.1431 - val_root_mean_squared_error: 0.0113 - val_loss: 
 1.2671e-04 - val_mape: 535661.8750 - val_accuracy: 0.1464
 Epoch 46/50
 621/621 [==============================] - 4s 7ms/step - root_mean_squared_error: 0.0196 - loss: 
 3.8274e-04 - mape: 1045924.3125 - accuracy: 0.1431 - val_root_mean_squared_error: 0.0101 - val_loss: 
 1.0106e-04 - val_mape: 587111.2500 - val_accuracy: 0.1464
 Epoch 47/50
 621/621 [==============================] - 4s 7ms/step - root_mean_squared_error: 0.0195 - loss: 
 3.7925e-04 - mape: 1038761.8125 - accuracy: 0.1431 - val_root_mean_squared_error: 0.0112 - val_loss: 
 1.2610e-04 - val_mape: 474619.3125 - val_accuracy: 0.1464
 Epoch 48/50
 621/621 [==============================] - 4s 7ms/step - root_mean_squared_error: 0.0194 - loss: 
 3.7453e-04 - mape: 1024884.4375 - accuracy: 0.1431 - val_root_mean_squared_error: 0.0106 - val_loss: 
 1.1254e-04 - val_mape: 537549.6250 - val_accuracy: 0.1464
 Epoch 49/50
 621/621 [==============================] - 4s 7ms/step - root_mean_squared_error: 0.0193 - loss: 
 3.7414e-04 - mape: 1033414.7500 - accuracy: 0.1431 - val_root_mean_squared_error: 0.0122 - val_loss: 
 1.4766e-04 - val_mape: 475745.0000 - val_accuracy: 0.1464
 Epoch 50/50
 621/621 [==============================] - 4s 7ms/step - root_mean_squared_error: 0.0194 - loss: 
 3.7510e-04 - mape: 1027084.1250 - accuracy: 0.1431 - val_root_mean_squared_error: 0.0094 - val_loss: 
 8.9167e-05 - val_mape: 506829.9062 - val_accuracy: 0.1464

Tranning graph

我很困惑如何提高 DNN 预测模型的准确性。如果有人给我建议，我将非常感谢。

【问题讨论】：

目标列具有连续值而不是离散值。所以本质上，这是一个回归问题，而不是分类问题。因此，这里的监控准确性没有多大意义。准确性在分类问题中很有用。另一方面，您会注意到您的loss 和val_loss 正在减少，这意味着您的模型正在学习。

@AdityaMishra 谢谢你的建议。是的，我的目标列是连续值，我专注于一个预测，而不是任何分类问题。此外，我的输出将是一个连续的形式。因此，在这种情况下，您可以向我建议该建模的合适参数。我已经上传了训练数据，你也可以查看我的模型。我对这个模型很困惑，所以我请求你检查它并给我建议。

【参考方案1】：

你的损失是MSE，就像你要解决的问题是回归一样。准确性是分类的指标，这就是您获得几乎没有变化的准确性的原因。如果您想使用回归，我建议您也使用 MSE 评估您的模型

如果您确实想要进行分类（我认为这对于您的连续目标值不是一个好主意），您需要将损失从 MSE 更改为 cross-entropy，无论是二进制还是分类，都取决于如果您正在解决二进制或多类分类任务

【讨论】：

谢谢@alift。是的，我专注于预测，而不是任何分类。此外，我的输出将是一个连续的形式。所以在这种情况下，我应该为这个建模选择什么参数。我已经上传了训练数据，你也可以查看我的模型。我对如何使用 MSE 函数评估模型感到困惑。为什么我必须使用MSE函数而不是Accuracy函数。