Tensorflow：保存模型并恢复单个权重

Posted 2023-05-07

【中文标题】Tensorflow：保存模型并恢复单个权重

【英文标题】：Tensorflow: Saving model and restoring individual weights

【发布时间】：2017-04-29 19:10:06

【问题描述】：

我的问题是：我保存了一个模型。现在我想更改模型（添加或删除一些层），但仍然导入我之前训练的权重。如何在初始化新部分的同时恢复模型的各个部分？

更具体一点：

我有一个 tensorflow 模型，其权重如下所示：

global_step = tf.get_variable("global_step", shape=[1],
                  trainable=False, initializer=tf.constant_initializer(1))

#TODO FUTURE: Time-video as input, for possible Seq2Seq model
#TODO: Add "regularizer=None"
Weights = 
    "W_Conv1": tf.get_variable("W_Conv1", shape=[3, 3, 1, 64],
            initializer=tf.random_normal_initializer(mean=0.00, stddev=0.01),
    ),
    "W_Conv2": tf.get_variable("W_Conv2", shape=[3, 3, 64, 64],
            initializer=tf.random_normal_initializer(mean=0.00, stddev=0.01),
    ),
    #not sure if we sum over all 64 channels?!
    "W_Local1": tf.get_variable("W_Local1", shape=[1, 16 * 8 * 64],
            initializer=tf.random_normal_initializer(mean=0.00, stddev=0.01),
    ),
    "W_Local2": tf.get_variable("W_Local2", shape=[1, 16 * 8 * 64],
            initializer=tf.random_normal_initializer(mean=0.00, stddev=0.01),
    ),
    "W_Affine1": tf.get_variable("W_Affine1", shape=[16*8*64, 512],
            initializer=tf.random_normal_initializer(mean=0.00, stddev=0.01),
    ),
    "W_Affine2": tf.get_variable("W_Affine2", shape=[512, 128],
            initializer=tf.random_normal_initializer(mean=0.00, stddev=0.01),
    ),
    "W_Affine3": tf.get_variable("W_Affine3", shape=[128, 10],
            initializer=tf.random_normal_initializer(mean=0.00, stddev=0.01),
    )


#TODO: Do local layers have a bias term?
Bias = 
    "b_Conv1": tf.get_variable("b_Conv1", shape=[1, 16, 8, 64],
            initializer=tf.random_normal_initializer(mean=0.00, stddev=0.01),
    ),
    "b_Conv2": tf.get_variable("b_Conv2", shape=[1, 16, 8, 64],
            initializer=tf.random_normal_initializer(mean=0.00, stddev=0.01),
    ),
    "b_Local1": tf.get_variable("b_Local1", shape=[1, 8192],
            initializer=tf.random_normal_initializer(mean=0.00, stddev=0.01),
    ),
    "b_Local2": tf.get_variable("b_Local2", shape=[1, 8192],
            initializer=tf.random_normal_initializer(mean=0.00, stddev=0.01),
    ),
    "b_Affine1": tf.get_variable("b_Affine1", shape=[1, 512],
            initializer=tf.random_normal_initializer(mean=0.00, stddev=0.01),
    ),
    "b_Affine2": tf.get_variable("b_Affine2", shape=[1, 128],
            initializer=tf.random_normal_initializer(mean=0.00, stddev=0.01),
    ),
    "b_Affine3": tf.get_variable("b_Affine3", shape=[1, 10],
            initializer=tf.random_normal_initializer(mean=0.00, stddev=0.01),
    )

假设我使用 tf.train.Saver() 保存了这个模型。 现在，我想运行这个模型。但是，在将来的某个时候，我可能想更改模型，但保留我目前训练的权重。如何单独恢复某些权重？ 更具体地说，如何再次恢复模型，如果我添加行

tmpval = tf.get_variable("new_var", shape=[1],
             initializer=tf.constant_initializer(0.1))

到上面定义的模型权重？

我是 tensorflow 的初学者，因此欢迎任何建议和想法。提前致谢！ :)

【参考方案1】：

我认为你的问题已经在 TF 教程'Choosing which variables to save and restore' 中介绍过

# Create some variables.
v1 = tf.Variable(..., name="v1")
v2 = tf.Variable(..., name="v2")
...
# Add ops to save and restore only 'v2' using the name "my_v2"
saver = tf.train.Saver("my_v2": v2)
# Use the saver object normally after that.
...

【讨论】：

这是否意味着：如果我将所有变量的字典传递给 tf.train.Saver，未初始化的将被初始化，其余的将被加载？