第二个时期的初始损失与第一个时期的最终损失并不一致。每个时期的损失和准确度保持不变

Posted 2023-03-12

【中文标题】第二个时期的初始损失与第一个时期的最终损失并不一致。每个时期的损失和准确度保持不变

【英文标题】：The second epoch's initial loss is not consistently related to the first epoch's final loss. The loss and the accuracy remain constant every epoch

【发布时间】：2022-01-24 06:44:30

【问题描述】：

第二个损失与第一个 epoch 并不一致。之后，每个初始损失在每个时期都保持不变。所有这些参数都保持不变。我有一些深度学习的背景，但这是我第一次实现自己的模型，所以我想直观地知道我的模型出了什么问题。数据集是裁剪后的人脸，有两个分类，每个分类有 300 张图片。非常感谢您的帮助。

import tensorflow as tf
from tensorflow import keras
from IPython.display import Image
import matplotlib.pyplot as plt
from keras.layers import ActivityRegularization
from keras.layers import Dropout
from tensorflow.keras.preprocessing.image import ImageDataGenerator

image_generator = ImageDataGenerator(
    featurewise_center=False, samplewise_center=False,
    featurewise_std_normalization=False, samplewise_std_normalization=False,
    rotation_range=0, width_shift_range=0.0, height_shift_range=0.0,
    brightness_range=None, shear_range=0.0, zoom_range=0.0, channel_shift_range=0.0,
    horizontal_flip=False, vertical_flip=False, rescale=1./255
)

image = image_generator.flow_from_directory('./util/untitled folder',batch_size=938)

x, y = image.next()
x_train = x[:500]
y_train = y[:500]
x_test = x[500:600]
y_test = y[500:600]

train_dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train)).batch(4)
test_dataset = tf.data.Dataset.from_tensor_slices((x_test, y_test)).batch(4)

plt.imshow(x_train[0])

def convolutional_model(input_shape):

    input_img = tf.keras.Input(shape=input_shape)
    x = tf.keras.layers.Conv2D(64, (7,7), padding='same')(input_img)
    x = tf.keras.layers.BatchNormalization(axis=3)(x)
    x = tf.keras.layers.ReLU()(x)
    x = tf.keras.layers.MaxPool2D(pool_size=(2, 2), strides=1, padding='same')(x)
    x = Dropout(0.5)(x)
    x = tf.keras.layers.Conv2D(128, (3, 3), padding='same', strides=1)(x)
    x = tf.keras.layers.ReLU()(x)
    x = tf.keras.layers.MaxPool2D(pool_size=(2, 2), padding='same', strides=4)(x)
    x = tf.keras.layers.Flatten()(x)
    x = ActivityRegularization(0.1,0.2)(x)
    outputs = tf.keras.layers.Dense(2, activation='softmax')(x)


    model = tf.keras.Model(inputs=input_img, outputs=outputs)
    return model


conv_model = convolutional_model((256, 256, 3))
conv_model.compile(loss=keras.losses.categorical_crossentropy,
                   optimizer=keras.optimizers.SGD(lr=1),
                   metrics=['accuracy'])
conv_model.summary()

conv_model.fit(train_dataset,epochs=100, validation_data=test_dataset)


Epoch 1/100
    2021-12-23 15:06:22.165763: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
    2021-12-23 15:06:22.172255: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
    125/125 [==============================] - ETA: 0s - loss: 804.6805 - accuracy: 0.48602021-12-23 15:06:50.936870: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
    125/125 [==============================] - 35s 275ms/step - loss: 804.6805 - accuracy: 0.4860 - val_loss: 0.7197 - val_accuracy: 0.4980
    Epoch 2/100
    125/125 [==============================] - 34s 270ms/step - loss: 0.7360 - accuracy: 0.4820 - val_loss: 0.7197 - val_accuracy: 0.4980
    Epoch 3/100
    125/125 [==============================] - 34s 276ms/step - loss: 0.7360 - accuracy: 0.4820 - val_loss: 0.7197 - val_accuracy: 0.4980


    

【参考方案1】：

由于您的损失和准确率恒定不变，因此您的网络很可能没有学到任何东西（因为您有两个类，它总是预测其中一个）。

最后一层的激活函数、损失函数和神经元个数都正确。

问题与加载图像的方式无关，而与学习率有关，即 1。在如此高的学习率下，网络不可能学习任何东西。

您应该从一个小得多的学习率开始，例如0.0001 或0.00001，然后如果性能仍然很差，请尝试调试数据加载过程。

【参考方案2】：

我很确定这与您加载数据的方式有关，更具体地说是x, y = image.next() 部分。如果您能够将数据从./util/untitled folder 拆分为分别具有训练和验证数据的单独文件夹，则可以在管道上使用与examples section on Tensorflow page 中相同的类型：

train_datagen = ImageDataGenerator(
    featurewise_center=False, 
    samplewise_center=False,
    featurewise_std_normalization=False, 
    samplewise_std_normalization=False,
    rotation_range=0, 
    width_shift_range=0.0, 
    height_shift_range=0.0,
    brightness_range=None, 
    shear_range=0.0, 
    zoom_range=0.0, 
    channel_shift_range=0.0,
    horizontal_flip=False, 
    vertical_flip=False, 
    rescale=1./255)
test_datagen = ImageDataGenerator(featurewise_center=False, 
    samplewise_center=False,
    featurewise_std_normalization=False, 
    samplewise_std_normalization=False,
    rotation_range=0, 
    width_shift_range=0.0, 
    height_shift_range=0.0,
    brightness_range=None, 
    shear_range=0.0, 
    zoom_range=0.0, 
    channel_shift_range=0.0,
    horizontal_flip=False, 
    vertical_flip=False, 
    rescale=1./255)
train_generator = train_datagen.flow_from_directory(
    'data/train',
    target_size=(256, 256),
    batch_size=4)
validation_generator = test_datagen.flow_from_directory(
    'data/validation',
    target_size=(256, 256),
    batch_size=4)
model.fit(
    train_generator,
    epochs=100,
    validation_data=validation_generator)

【讨论】：

第二个epoch的loss还是下降的太突然了。 Epoch 1/100 2021-12-23 16:47:11.849357: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] 启用了 device_type GPU 的插件优化器。 169/169 [==============================] - 38s 223ms/步 - 损失：531.8750 - 精度：0.4852 Epoch 2 /100 169/169 [===============================] - 38s 226ms/步 - 损失：0.6932 - 准确度：0.5059