在基础 Tensorflow 2.0 中运行简单回归
Posted
技术标签:
【中文标题】在基础 Tensorflow 2.0 中运行简单回归【英文标题】:Running simple regression in base Tensorflow 2.0 【发布时间】:2019-09-08 03:06:30 【问题描述】:我正在学习 Tensorflow 2.0,我认为在 Tensorflow 中实现最基本的简单线性回归是一个好主意。不幸的是,我遇到了几个问题,我想知道这里是否有人可以提供帮助。
考虑以下设置:
import tensorflow as tf # 2.0.0-alpha0
import numpy as np
x_data = np.random.randn(2000, 1)
w_real = [0.7] # coefficients
b_real = -0.2 # global bias
noise = np.random.randn(1, 2000) * 0.5 # level of noise
y_data = np.matmul(w_real, x_data.T) + b_real + noise
现在开始定义模型:
# modelling this data with tensorflow (manually!)
class SimpleRegressionNN(tf.keras.Model):
def __init__(self):
super(SimpleRegressionNN, self).__init__()
self.input_layer = tf.keras.layers.Input
self.output_layer = tf.keras.layers.Dense(1)
def call(self, data_input):
model = self.input_layer(data_input)
model = self.output_layer(model)
# open question: how to account for the intercept/bias term?
# Ideally, we'd want to generate preds as matmult(X,W) + b
return model
nn_regressor = SimpleRegressionNN()
reg_loss = tf.keras.losses.MeanSquaredError()
reg_optimiser = tf.keras.optimizers.SGD(0.1)
metric_accuracy = tf.keras.metrics.mean_squared_error
# define forward step
@tf.function
def train_step(x_sample, y_sample):
with tf.GradientTape() as tape:
predictions = nn_regressor(x_sample)
loss = reg_loss(y_sample, predictions)
gradients = tape.gradient(loss, nn_regressor.trainable_variables) # had to indent this!
reg_optimiser.apply_gradients(zip(gradients, nn_regressor.trainable_variables))
metric_accuracy(y_sample, predictions)
#%%
# run the model
for epoch in range(10):
for x_point, y_point in zip(x_data.T[0], y_data[0]): # batch of 1
train_step(x_sample=x_point, y_sample=y_point)
print("MSE: ".format(metric_accuracy.result()))
很遗憾,我收到以下错误:
TypeError: You are attempting to use Python control flow in a layer that was not declared to be dynamic. Pass `dynamic=True` to the class constructor.
Encountered error:
"""
Tensor objects are only iterable when eager execution is enabled. To iterate over this tensor use tf.map_fn.
"""
完整的错误输出在这里:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/anaconda3/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, inputs, *args, **kwargs)
611 inputs)) as auto_updater:
--> 612 outputs = self.call(inputs, *args, **kwargs)
613 auto_updater.set_outputs(outputs)
<ipython-input-5-8464ad8bcf07> in call(self, data_input)
7 def call(self, data_input):
----> 8 model = self.input_layer(data_input)
9 model = self.output_layer(model)
/anaconda3/lib/python3.6/site-packages/tensorflow/python/keras/engine/input_layer.py in Input(shape, batch_size, name, dtype, sparse, tensor, **kwargs)
232 sparse=sparse,
--> 233 input_tensor=tensor)
234 # Return tensor including `_keras_history`.
/anaconda3/lib/python3.6/site-packages/tensorflow/python/keras/engine/input_layer.py in __init__(self, input_shape, batch_size, dtype, input_tensor, sparse, name, **kwargs)
93 if input_shape is not None:
---> 94 batch_input_shape = (batch_size,) + tuple(input_shape)
95 else:
/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in __iter__(self)
448 raise TypeError(
--> 449 "Tensor objects are only iterable when eager execution is "
450 "enabled. To iterate over this tensor use tf.map_fn.")
TypeError: Tensor objects are only iterable when eager execution is enabled. To iterate over this tensor use tf.map_fn.
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
<ipython-input-22-e1bde858b0fc> in <module>()
3 #train_step(x_sample=x_data.T[0], y_sample=y_data[0])
4 for x_point, y_point in zip(x_data.T[0], y_data[0]):
----> 5 train_step(x_sample=x_point, y_sample=y_point)
6 print("MSE: ".format(metric_accuracy.result()))
7
/anaconda3/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py in __call__(self, *args, **kwds)
416 # In this case we have not created variables on the first call. So we can
417 # run the first trace but we should fail if variables are created.
--> 418 results = self._stateful_fn(*args, **kwds)
419 if self._created_variables:
420 raise ValueError("Creating variables on a non-first call to a function"
/anaconda3/lib/python3.6/site-packages/tensorflow/python/eager/function.py in __call__(self, *args, **kwargs)
1285 def __call__(self, *args, **kwargs):
1286 """Calls a graph function specialized to the inputs."""
-> 1287 graph_function, args, kwargs = self._maybe_define_function(args, kwargs)
1288 return graph_function._filtered_call(args, kwargs) # pylint: disable=protected-access
1289
/anaconda3/lib/python3.6/site-packages/tensorflow/python/eager/function.py in _maybe_define_function(self, args, kwargs)
1609 relaxed_arg_shapes)
1610 graph_function = self._create_graph_function(
-> 1611 args, kwargs, override_flat_arg_shapes=relaxed_arg_shapes)
1612 self._function_cache.arg_relaxed[rank_only_cache_key] = graph_function
1613
/anaconda3/lib/python3.6/site-packages/tensorflow/python/eager/function.py in _create_graph_function(self, args, kwargs, override_flat_arg_shapes)
1510 arg_names=arg_names,
1511 override_flat_arg_shapes=override_flat_arg_shapes,
-> 1512 capture_by_value=self._capture_by_value),
1513 self._function_attributes)
1514
/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes)
692 converted_func)
693
--> 694 func_outputs = python_func(*func_args, **func_kwargs)
695
696 # invariant: `func_outputs` contains only Tensors, IndexedSlices,
/anaconda3/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py in wrapped_fn(*args, **kwds)
315 # __wrapped__ allows AutoGraph to swap in a converted function. We give
316 # the function a weak reference to itself to avoid a reference cycle.
--> 317 return weak_wrapped_fn().__wrapped__(*args, **kwds)
318 weak_wrapped_fn = weakref.ref(wrapped_fn)
319
/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
684 optional_features=autograph_options,
685 force_conversion=True,
--> 686 ), args, kwargs)
687
688 # Wrapping around a decorator allows checks like tf_inspect.getargspec
/anaconda3/lib/python3.6/site-packages/tensorflow/python/autograph/impl/api.py in converted_call(f, owner, options, args, kwargs)
390 return _call_unconverted(f, args, kwargs)
391
--> 392 result = converted_f(*effective_args, **kwargs)
393
394 # The converted function's closure is simply inserted into the function's
/var/folders/8_/pl9fgq297ld3b7kgy5tmvf700000gn/T/tmpluzodr7d.py in tf__train_step(x_sample, y_sample)
2 def tf__train_step(x_sample, y_sample):
3 with tf.GradientTape() as tape:
----> 4 predictions = ag__.converted_call(nn_regressor, None, ag__.ConversionOptions(recursive=True, verbose=0, strip_decorators=(tf.function, defun, ag__.convert, ag__.do_not_convert, ag__.converted_call), force_conversion=False, optional_features=(), internal_convert_user_code=True), (x_sample,), )
5 loss = ag__.converted_call(reg_loss, None, ag__.ConversionOptions(recursive=True, verbose=0, strip_decorators=(tf.function, defun_1, ag__.convert, ag__.do_not_convert, ag__.converted_call), force_conversion=False, optional_features=(), internal_convert_user_code=True), (y_sample, predictions), )
6 gradients = ag__.converted_call('gradient', tape, ag__.ConversionOptions(recursive=True, verbose=0, strip_decorators=(tf.function, defun_2, ag__.convert, ag__.do_not_convert, ag__.converted_call), force_conversion=False, optional_features=(), internal_convert_user_code=True), (loss, nn_regressor.trainable_variables), )
/anaconda3/lib/python3.6/site-packages/tensorflow/python/autograph/impl/api.py in converted_call(f, owner, options, args, kwargs)
265
266 if not options.force_conversion and conversion.is_whitelisted_for_graph(f):
--> 267 return _call_unconverted(f, args, kwargs)
268
269 # internal_convert_user_code is for example turned off when issuing a dynamic
/anaconda3/lib/python3.6/site-packages/tensorflow/python/autograph/impl/api.py in _call_unconverted(f, args, kwargs)
186 return f.__self__.call(args, kwargs)
187
--> 188 return f(*args, **kwargs)
189
190
/anaconda3/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, inputs, *args, **kwargs)
623 'dynamic. Pass `dynamic=True` to the class '
624 'constructor.\nEncountered error:\n"""\n' +
--> 625 exception_str + '\n"""')
626 raise
627 else:
TypeError: You are attempting to use Python control flow in a layer that was not declared to be dynamic. Pass `dynamic=True` to the class constructor.
Encountered error:
"""
Tensor objects are only iterable when eager execution is enabled. To iterate over this tensor use tf.map_fn.
"""
麻烦的是,2.0默认设置为eager execution!
除了这个问题,我还有几个问题:
-
在这里解释截距项的最佳方法是什么?
一般方法是否合理,或者我在这里做了什么奇怪的事情? (忽略批量大小和我必须验证数据的事实,这只是一个玩具示例)
非常感谢!
【问题讨论】:
(1)
tf.keras.layers.Dense()
默认添加拦截。错误点的确切线是什么?
感谢您的快速回复。恐怕定位错误的根源并不是那么简单。我已经包含了整个错误输出,希望对您有所帮助。我刚刚尝试单独运行train_step
函数,它给了我同样的错误,我可以缩小范围。
【参考方案1】:
我有以下言论:
您的SimpleRegression
模型中不需要Input
层。此外,不要通过“model
”名称调用层的张量输出(就像在 call()
方法中所做的那样)。这真是令人困惑。
您没有将正确的形状传递给您的train_step
函数。它预计在您通过时收到(n_samples, input_dim)
(input_dim
, )。
请记住,在tensorflow
中,张量的第一维始终是批量大小(即样本数)。总是这样使用它,而不是移调。
为什么要调用metric_accuracy = tf.keras.metrics.mean_squared_error
准确度?你有一个回归问题,回归中没有准确性。还有,为什么要定义两次,计算两次mse
?
如果您使用tf.convert_to_tensor()
转换数据,执行速度会更快。
函数train_step()
执行向前和向后传球,而不仅仅是向前传球。
为玩具示例使用小型数据集(2-10 个样本,而不是 2000 个),尤其是在您不知道自己的代码是否有效的情况下!
您的函数train_step()
没有返回任何内容,您希望如何打印mse
损失的值。
这是您的代码的更正版本:
import tensorflow as tf # 2.0.0-alpha0
import numpy as np
x_data = np.random.randn(5, 2)
w_real = 0.7 # coefficients
b_real = -0.2 # global bias
noise = np.random.randn(5, 2) * 0.01 # level of noise
y_data = w_real * x_data + b_real + noise
class SimpleRegressionNN(tf.keras.Model):
def __init__(self):
super(SimpleRegressionNN, self).__init__()
self.output_layer = tf.keras.layers.Dense(1, input_shape=(2, ))
def call(self, data_input):
result = self.output_layer(data_input)
return result
reg_loss = tf.keras.losses.MeanSquaredError()
reg_optimiser = tf.keras.optimizers.SGD(0.1)
nn_regressor = SimpleRegressionNN()
@tf.function
def train_step(x_sample, y_sample):
with tf.GradientTape() as tape:
predictions = nn_regressor(x_sample)
loss = reg_loss(y_sample, predictions)
gradients = tape.gradient(loss, nn_regressor.trainable_variables) # had to indent this!
reg_optimiser.apply_gradients(zip(gradients, nn_regressor.trainable_variables))
return loss
for x_point, y_point in zip(x_data, y_data): # batch of 1
x_point, y_point = tf.convert_to_tensor([x_point]), tf.convert_to_tensor([y_point])
mse = train_step(x_sample=x_point, y_sample=y_point)
print("MSE: ".format(mse.numpy()))
【讨论】:
弗拉德,非常感谢!我正在关注一些可以在网上找到的示例(但在 TF 2.0 上没有太多示例),这就是为什么我的代码中有一些奇怪之处。您的示例非常有道理,再次感谢您! 很高兴它有帮助。我已经纠正了我的代码中的小错误。最好在GradientTape
上下文之外计算和应用梯度(如果您不打算计算二阶导数)。这不是一个错误,但如果您在 GradientTape
上下文中执行此操作,它会将这些操作记录到缓冲区中,这会增加不必要的开销。以上是关于在基础 Tensorflow 2.0 中运行简单回归的主要内容,如果未能解决你的问题,请参考以下文章
社区分享 | Spark 玩转 TensorFlow 2.0
在 Tensorflow 2.0 中是不是有更简单的方法来执行模型层?