Keras ImageDataGenerator 不处理符号链接文件

Posted 2023-02-16

【中文标题】Keras ImageDataGenerator 不处理符号链接文件

【英文标题】：Keras ImageDataGenerator not handling symlink files

【发布时间】：2022-01-15 07:54:40

【问题描述】：

我正在尝试使用 tensorflow.keras 在胸部 X 射线图像上训练 DenseNet121 模型，并使用 ImageDataGenerator 进行增强。我的文件目录包含指向图像的符号链接，我认为这些图像的格式设置为 ImageDataGenerator 的正确格式：

火车

正常

异常

值

正常

异常

但是，当我调用 model.fit() 时，它会抛出 FileNotFoundError: [Errno 2] No such file or directory: '.\\Train\\Normal\\00017275_014.png' 这是一个符号链接文件。 .flow_from_directory(follow_links = True) 没有解决问题。此外，使用该路径调用 os.islink() 会返回 True。

另外：调用imagedatagenerator返回： Found 84090 images belonging to 2 classes. Found 28030 images belonging to 2 classes.

有什么建议吗？代码如下：

from tensorflow.keras.applications.densenet import preprocess_input
from tensorflow.keras import Model,layers
from tensorflow.keras.preprocessing.image import ImageDataGenerator

from tensorflow.keras.optimizers import Adam, SGD
from tensorflow.keras.metrics import binary_accuracy
from tensorflow.keras.losses import binary_crossentropy

batch_size = 64

train_datagen = ImageDataGenerator(
    preprocessing_function = preprocess_input,
    brightness_range = [0.75, 1.25],
    horizontal_flip=True,
)

train_generator = train_datagen.flow_from_directory(
    directory = '.\\Train',
    color_mode = 'rgb',
    classes = ['Normal', 'Abnormal'],
    class_mode = 'binary',
    batch_size = batch_size,
    target_size = (224,224),
    follow_links=True,
)

val_datagen = ImageDataGenerator(
    preprocessing_function = preprocess_input,
)

val_generator = val_datagen.flow_from_directory(
    directory = '.\\Val',
    color_mode = 'rgb',
    class_mode = 'binary',
    classes = ['Normal', 'Abnormal'],
    batch_size = batch_size,
    target_size = (224,224),
    follow_links = True,
)

from tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping

model_name = "Imagenet DenseNet121 on NIH full dataset 375 locked brightness flip.h5"
callback_checkpoint = [
    EarlyStopping(monitor = 'val_loss', patience = 10, verbose = 1),
    ModelCheckpoint(model_name,
                    verbose = 1,
                    monitor = 'val_loss',
                    save_best_only = True,
                   )
]

model.compile(
    optimizer = Adam(),
    #optimizer = SGD(learning_rate = 0.001, momentum = 0.9, decay = 0.0001),
    loss = 'binary_crossentropy',
    metrics = ['binary_accuracy'],
)

history = model.fit(
        train_generator,
        steps_per_epoch=1250,
        epochs=50,
        validation_data=val_generator,
        validation_steps=437,
        callbacks = [callback_checkpoint],
)

`os.path.islink((os.path.join(os.getcwd(), “火车”， “普通的”， "00017275_014.png")))

是的`

【参考方案1】：

至少对于pathlib.Path，带有点和双反斜杠的组合表示法无效。我想这也是这里的问题。尝试使用正斜杠。而不是directory = ".\\Val"试试

directory = "./Val"

或者干脆

directory = "Val"