在 tensorflow 2.4 中使用 sampled_softmax 时无法将符号 Keras 输入/输出转换为 numpy 数组 TypeError
Posted
技术标签:
【中文标题】在 tensorflow 2.4 中使用 sampled_softmax 时无法将符号 Keras 输入/输出转换为 numpy 数组 TypeError【英文标题】:Cannot convert a symbolic Keras input/output to a numpy array TypeError when using sampled_softmax in tensorflow 2.4 【发布时间】:2021-03-29 15:54:22 【问题描述】:我正在尝试使用带有 Keras 的 TF2.4 并使用 tf.nn.sampled_softmax_loss 来训练一个词嵌入分类器。但是,在调用模型的 fit 方法时,会出现“无法将符号 Keras 输入/输出转换为 numpy 数组”的 TypeError。请帮助我修复错误或使用替代方法进行候选抽样。
import tensorflow as tf
import numpy as np
TextVectorization = tf.keras.layers.experimental.preprocessing.TextVectorization
class SampledSoftmaxLoss: #(tf.keras.losses.Loss):
def __init__(self, model, n_classes):
self.model = model
output_layer = model.layers[-1]
self.input = output_layer.input
self.weights = output_layer.weights
self.n_classes = n_classes
def loss(self, y_true, y_pred, **kwargs):
labels = tf.argmax(y_true, axis=1)
labels = tf.expand_dims(labels, -1)
loss = tf.nn.sampled_softmax_loss(
weights=self.weights[0],
biases=self.weights[1],
labels=labels,
inputs=self.input,
num_sampled = 3,
num_classes = self.n_classes
)
return loss
max_features = 50 # Maximum vocab size.
max_len = 10 # Sequence length to pad the outputs to.
embedding_dims = 5
input_data = np.array([
"Python Machine Learning",
"Data Science from Scratch: First Principles with Python",
"Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques for Building Intelligent Systems",
"Introduction to Machine Learning with Python: A Guide for Data Scientists",
"Vital Introduction to Machine Learning with Python: Best Practices to Improve and Optimize Machine Learning Systems and Algorithms",
"Machine Learning in Python: Essential Techniques for Predictive Analysis",
"Python Data Science Handbook: Essential Tools for Working with Data",
"Introducing Data Science: Big Data, Machine Learning, and more, using Python tools",
"Real-World Machine Learning"])
labels_one_hot = []
for i in range(len(input_data)):
labels = np.random.randint(0, 6, max_features)
labels[labels>1] = 1
labels_one_hot.append(labels)
labels_one_hot = np.array(labels_one_hot)
# Create the text categorical encoding layer.
vectorize_layer = TextVectorization(
max_tokens=max_features,
output_mode='int',
output_sequence_length=max_len)
vectorize_layer.adapt(text_dataset)
inp = tf.keras.Input(shape=(1,), dtype=tf.string)
idxs = vectorize_layer(inp)
embed = tf.keras.layers.Embedding(max_features + 1, embedding_dims,input_length=max_len)(idxs)
flat = tf.keras.layers.Flatten()(embed)
out = tf.keras.layers.Dense(labels_one_hot.shape[1])(flat)
model = tf.keras.models.Model(inp, out)
softmax = SampledSoftmaxLoss(model, labels_one_hot.shape[1])
model.compile('adam', loss=softmax.loss)
model.summary()
model.fit(input_data, labels_one_hot)
model.predict(input_data)
这是日志:
INFO:tensorflow:Assets written to: model/assets
Model: "model_5"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_84 (InputLayer) [(None, 1)] 0
_________________________________________________________________
text_vectorization_83 (TextV (None, 10) 0
_________________________________________________________________
embedding_83 (Embedding) (None, 10, 5) 255
_________________________________________________________________
flatten_25 (Flatten) (None, 50) 0
_________________________________________________________________
dense_69 (Dense) (None, 50) 2550
=================================================================
Total params: 2,805
Trainable params: 2,805
Non-trainable params: 0
_________________________________________________________________
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-87-c35b8d1321a1> in <module>()
106 model.compile('adam', loss=softmax.loss)
107 model.summary()
--> 108 model.fit(input_data, labels_one_hot)
109 # Now, the model can map strings to integers, and you can add an embedding
110 # layer to map these integers to learned embeddings.
9 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
975 except Exception as e: # pylint:disable=broad-except
976 if hasattr(e, "ag_error_metadata"):
--> 977 raise e.ag_error_metadata.to_exception(e)
978 else:
979 raise
TypeError: in user code:
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:805 train_function *
return step_function(self, iterator)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:795 step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
/usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:1259 run
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2730 call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:3417 _call_for_each_replica
return fn(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:788 run_step **
outputs = model.train_step(data)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:756 train_step
y, y_pred, sample_weight, regularization_losses=self.losses)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/compile_utils.py:238 __call__
total_loss_metric_value, sample_weight=batch_dim)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/utils/metrics_utils.py:90 decorated
update_op = update_state_fn(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/metrics.py:177 update_state_fn
return ag_update_state(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/metrics.py:364 update_state **
sample_weight, values)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/weights_broadcast_ops.py:155 broadcast_weights
values = ops.convert_to_tensor(values, name="values")
/usr/local/lib/python3.6/dist-packages/tensorflow/python/profiler/trace.py:163 wrapped
return func(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:1540 convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/constant_op.py:339 _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/constant_op.py:265 constant
allow_broadcast=True)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/constant_op.py:283 _constant_impl
allow_broadcast=allow_broadcast))
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/tensor_util.py:435 make_tensor_proto
values = np.asarray(values)
/usr/local/lib/python3.6/dist-packages/numpy/core/_asarray.py:83 asarray
return array(a, dtype, copy=False, order=order)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/keras_tensor.py:274 __array__
'Cannot convert a symbolic Keras input/output to a numpy array. '
TypeError: Cannot convert a symbolic Keras input/output to a numpy array. This error may indicate that you're trying to pass a symbolic value to a NumPy call, which is not supported. Or, you may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model.
【问题讨论】:
你检查this question了吗?可能与您的问题有关 这可能是由于混合了 numpy 和 Tensorflow ops 数据类型等输入。您可以按照此链接和提到的教程检查TextVectorization
在不同解决方案中的用法。 tensorflow.org/api_docs/python/tf/keras/layers/experimental/…
【参考方案1】:
出现此错误的主要原因之一是您可能在 TensorFlow 之前安装了 Keras。 如果是这种情况,重新启动内核就可以了
【讨论】:
根本不行,我只安装了Tensorflow,用的是tensorflow包里的kreas【参考方案2】:错误是由您的自定义损失函数引起的。您应该禁用 TF 急切执行模式。
from tensorflow.python.framework.ops import disable_eager_execution
disable_eager_execution()
另外,在model.compile(...)
中使用experimental_run_tf_function=False
【讨论】:
这不是解决方案,在我的情况下,我在 GPU 上运行带有 LSTM 层的模型,一旦我禁用了急切执行,另一个错误是 LSTM 无法使用 GPU,它不遵守标准 experimental_run_tf_function=False in model.compile(...) 可以在没有 disable_eager_execution() 的情况下提供帮助。它对你有什么影响吗? 我遇到了同样的问题。disable_eager_execution()
有效,但使用后出现前置条件错误。【参考方案3】:
以下解决方案让我有点头疼,但我不确定它为什么有效。 我是 DL 新手,我只是想提供帮助。
试试:
del model
之前:
model = tf.keras.models.Model(inp, out)
我知道我可能错了,欢迎教育我,我愿意学习:)。
【讨论】:
以上是关于在 tensorflow 2.4 中使用 sampled_softmax 时无法将符号 Keras 输入/输出转换为 numpy 数组 TypeError的主要内容,如果未能解决你的问题,请参考以下文章
带有 TensorFlow 2.4+ 错误的 SHAP DeepExplainer
如何使用 Tensorflow V.2.4 RTX 2070 Super Ubuntu 18.04 安装 Cuda 10.1
如何强制 keras 使用 tensorflow GPU 后端