使用预训练 vgg19 tensorflow，Keras 在 CNN 自动编码器中定义自定义损失（感知损失）
Posted
技术标签:
【中文标题】使用预训练 vgg19 tensorflow，Keras 在 CNN 自动编码器中定义自定义损失（感知损失）【英文标题】：Define custom loss (perceptual loss) in CNN autoencoder with pre-train vgg19 tensorflow,Keras 【发布时间】：2021-04-05 15:02:30 【问题描述】：我想在 keras 中构建的自动编码器中定义 perceptual_loss。 我的自动编码器是这样的：
编码器：
input_encoder = Input((32,32,3),name = 'encoder_input')
encoder = Conv2D(16,(3,3),activation = 'relu',name = 'encoder_layer1')(input_encoder)
encoder = Flatten(name = 'encoder_layer2')(encoder)
latent_encoding = Dense(128 , activation = 'relu', name = 'encoder_layer3')(encoder)
Encoder = Model(inputs= [input_encoder], outputs=[latent_encoding],name = 'Encoder')
解码器：
input_decoder = Input(128,name = 'decoder_input')
decoder = Reshape((1, 1, 128),name = 'decoder_layer1')(input_decoder)
decoder = Conv2DTranspose(64, (2,2), activation='relu' , name = 'decoder_layer2')(decoder)
decoder = UpSampling2D(8 ,name = 'decoder_layer3')(decoder)
decoder = Conv2DTranspose(3, (9,9), activation='relu' , name = 'decoder_layer2')(decoder)
Decoder = Model(inputs= [input_decoder], outputs=[decoder ],name = 'Decoder')
自动编码器：
input = Input((32,32,3),name = 'input')
latent = Encoder(input)
output = Decoder(latent)
AE = Model(inputs= [input], outputs=[output ],name = 'AE')
现在我用预训练 vgg19 定义新的损失函数 perceptual_loss 像这样我得到输入图像并重建图像以预训练 vgg19 并从 vgg19 的某个层获得结果，然后我使用两个向量的减法作为 vgg19 中该层的误差然后我使用层误差的加权和来计算总误差：
selected_layers = ['block1_conv1', 'block2_conv2',"block3_conv3" ,'block4_conv3','block5_conv4']
selected_layer_weights = [1.0, 4.0 , 4.0 , 8.0 , 16.0]
def perceptual_loss(input_image , reconstruct_image):
vgg = VGG19(weights='imagenet', include_top=False, input_shape=(32,32,3))
vgg.trainable = False
outputs = [vgg.get_layer(l).output for l in selected_layers]
model = Model(vgg.input, outputs)
h1_list = model(input_image)
h2_list = model(reconstruct_image)
rc_loss = 0.0
for h1, h2, weight in zip(h1_list, h2_list, selected_layer_weights):
h1 = K.batch_flatten(h1)
h2 = K.batch_flatten(h2)
rc_loss = rc_loss + weight * K.sum(K.square(h1 - h2), axis=-1)
return rc_loss
然后我编译AE：
rmsprop = RMSprop(learning_rate=0.00025)
AE.compile(loss= perceptual_loss, optimizer= rmsprop)
但是当我想要适合 AE 时：
history = AE.fit(train_images, train_images,
epochs= 2,
verbose=1)
我得到错误
ValueError: tf.function-decorated 函数试图在非首次调用时创建变量。
请帮助我。谢谢
更新：
我通过@Mr. 的回答更新了损失函数。例如，但我得到新的错误： 现在我有了损失函数：
'''
define perceptual_loss
'''
selected_layers = ['block1_conv1', 'block2_conv2',"block3_conv3" ,'block4_conv3','block5_conv4']
selected_layer_weights = [1.0, 4.0 , 4.0 , 8.0 , 16.0]
vgg = VGG19(weights='imagenet', include_top=False, input_shape=(32,32,3))
vgg.trainable = False
outputs = [vgg.get_layer(l).output for l in selected_layers]
model = Model(vgg.input, outputs)
def perceptual_loss(input_image , reconstruct_image):
h1_list = model(input_image)
h2_list = model(reconstruct_image)
rc_loss = 0.0
for h1, h2, weight in zip(h1_list, h2_list, selected_layer_weights):
h1 = K.batch_flatten(h1)
h2 = K.batch_flatten(h2)
rc_loss = rc_loss + weight * K.sum(K.square(h1 - h2), axis=-1)
return rc_loss
我收到新错误：
ValueError Traceback (most recent call last)
<ipython-input-16-3133696ab8be> in <module>()
----> 1 VAE.fit(train_images[:5],train_images[:5],epochs=2,verbose=1)
2 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
1098 _r=1):
1099 callbacks.on_train_batch_begin(step)
-> 1100 tmp_logs = self.train_function(iterator)
1101 if data_handler.should_sync:
1102 context.async_wait()
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py in __call__(self, *args, **kwds)
826 tracing_count = self.experimental_get_tracing_count()
827 with trace.Trace(self._name) as tm:
--> 828 result = self._call(*args, **kwds)
829 compiler = "xla" if self._experimental_compile else "nonXla"
830 new_tracing_count = self.experimental_get_tracing_count()
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py in _call(self, *args, **kwds)
862 results = self._stateful_fn(*args, **kwds)
863 if self._created_variables:
--> 864 raise ValueError("Creating variables on a non-first call to a function"
865 " decorated with tf.function.")
866 return results
ValueError: Creating variables on a non-first call to a function decorated with tf.function.
更新 2
正如@Navid 所说，我在损失函数之前添加了@tf.function，错误就消失了！
'''
define perceptual_loss
'''
selected_layers = ['block1_conv1', 'block2_conv2',"block3_conv3" ,'block4_conv3','block5_conv4']
selected_layer_weights = [1.0, 4.0 , 4.0 , 8.0 , 16.0]
vgg = VGG19(weights='imagenet', include_top=False, input_shape=(32,32,3))
vgg.trainable = False
outputs = [vgg.get_layer(l).output for l in selected_layers]
model = Model(vgg.input, outputs)
@tf.function
def perceptual_loss(input_image , reconstruct_image):
h1_list = model(input_image)
h2_list = model(reconstruct_image)
rc_loss = 0.0
for h1, h2, weight in zip(h1_list, h2_list, selected_layer_weights):
h1 = K.batch_flatten(h1)
h2 = K.batch_flatten(h2)
rc_loss = rc_loss + weight * K.sum(K.square(h1 - h2), axis=-1)
return rc_loss ```
【问题讨论】：
【参考方案1】：只需在损失函数之外创建模型，并在定义损失函数之前使用@tf.function。
【讨论】：
【参考方案2】：您不应该在损失函数中创建模型，而是应该执行以下操作：
selected_layers = ['block1_conv1', 'block2_conv2',"block3_conv3" ,'block4_conv3','block5_conv4']
selected_layer_weights = [1.0, 4.0 , 4.0 , 8.0 , 16.0]
vgg = VGG19(weights='imagenet', include_top=False, input_shape=(32,32,3))
vgg.trainable = False
outputs = [vgg.get_layer(l).output for l in selected_layers]
model = Model(vgg.input, outputs)
@tf.function
def perceptual_loss(input_image , reconstruct_image):
h1_list = model(input_image)
h2_list = model(reconstruct_image)
rc_loss = 0.0
for h1, h2, weight in zip(h1_list, h2_list, selected_layer_weights):
h1 = K.batch_flatten(h1)
h2 = K.batch_flatten(h2)
rc_loss = rc_loss + weight * K.sum(K.square(h1 - h2), axis=-1)
return rc_loss
【讨论】：
我按照你说的更新了代码，但是得到了一个与之前的错误非常相似的新错误。 我在损失函数和问题解决之前添加了@tf.function。谢谢。
@alish 我的错误，忘记了，tf.function 将 Python 函数转换为其图形表示以加快计算速度，因此您可以在定义损失函数时使用它以上是关于使用预训练 vgg19 tensorflow，Keras 在 CNN 自动编码器中定义自定义损失（感知损失）的主要内容，如果未能解决你的问题，请参考以下文章