错误代码OpenAI权衡规范化get_variable tf1.4

Question

我正在尝试使用实现权重规范化的OpenAI源代码（Saliman的论文）。 https://github.com/openai/weightnorm/tree/master/tensorflow

该代码在tf 1.1上运行良好。但是我不能让它在1.4上工作......

初始化期间存在一个错误（数据驱动的权重初始化）。似乎tf在某些点之后无法创建变量。它创建非常缓慢的第一个权重，然后冻结。这是我试图在tf 1.4上使用的代码：

@add_arg_scope
def conv2d(x, num_filters, filter_size=[3,3], stride=[1,1], pad='SAME', nonlinearity=None, init_scale=1., counters={}, init=False, ema=None, **kwargs):
    ''' convolutional layer '''
    name = get_name('conv2d', counters)
    with tf.variable_scope(name):
        if init:
            # data based initialization of parameters
            V = tf.get_variable('V', filter_size+[int(x.get_shape()[-1]),num_filters], tf.float32, tf.random_normal_initializer(0, 0.05), trainable=True)
            V_norm = tf.nn.l2_normalize(V.initialized_value(), [0,1,2])
            x_init = tf.nn.conv2d(x, V_norm, [1]+stride+[1], pad)
            m_init, v_init = tf.nn.moments(x_init, [0,1,2])
            scale_init = init_scale/tf.sqrt(v_init + 1e-8)
            g = tf.get_variable('g', dtype=tf.float32, initializer=scale_init, trainable=True)
            b = tf.get_variable('b', dtype=tf.float32, initializer=-m_init*scale_init, trainable=True)
            x_init = tf.reshape(scale_init,[1,1,1,num_filters])*(x_init-tf.reshape(m_init,[1,1,1,num_filters]))
            if nonlinearity is not None:
                x_init = nonlinearity(x_init)
            return x_init

它成功创建了第一层，但一段时间后它永远冻结。如果我停止该程序，这是我得到的消息：

 File "train_gan_cifar.py", line 309, in <module>
    tf.app.run()
  File "/home/bruno/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 48, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "train_gan_cifar.py", line 100, in main
    dis(unl, is_training_pl, init=True)
  File "/home/bruno/ssl-project/cifar_gan.py", line 35, in discriminator
    x = nn.conv2d(x, 192, nonlinearity=leakyReLu, init=init, counters=counter)
  File "/home/bruno/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 181, in func_with_args
    return func(*args, **current_args)
  File "/home/bruno/ssl-project/nn.py", line 192, in conv2d
    g = tf.get_variable('g', dtype=tf.float32, initializer=scale_init, trainable=True,validate_shape=False)
  File "/home/bruno/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 1203, in get_variable
    constraint=constraint)
  File "/home/bruno/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 1092, in get_variable
    constraint=constraint)
  File "/home/bruno/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 425, in get_variable
    constraint=constraint)
  File "/home/bruno/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 394, in _true_getter
    use_resource=use_resource, constraint=constraint)
  File "/home/bruno/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 805, in _get_single_variable
    constraint=constraint)
  File "/home/bruno/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 213, in __init__
    constraint=constraint)
  File "/home/bruno/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 345, in _init_from_args
    self._build_initializer_expr(self._initial_value),
  File "/home/bruno/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 777, in _build_initializer_expr
    new_op = self._build_initializer_expr(initial_value.op)
  File "/home/bruno/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 796, in _build_initializer_expr
    new_tensor = self._build_initializer_expr(tensor)
  File "/home/bruno/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 777, in _build_initializer_expr
    new_op = self._build_initializer_expr(initial_value.op)
  File "/home/bruno/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 796, in _build_initializer_expr
    new_tensor = self._build_initializer_expr(tensor)
  File "/home/bruno/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 777, in _build_initializer_expr
    new_op = self._build_initializer_expr(initial_value.op)
  File "/home/bruno/anaconda3/envs/tensorflow/lib/python3.5/sit

最后几行重复多次。