one_shot_iterator，占位符，无法捕捉占位符

Posted 2021-04-13

我试图从一个数据集进行one_shot_iterator。

我使用占位符使用更少的GPU内存和期待，我只需要初始化迭代器只有一次。

但我得到的错误：

Traceback (most recent call last):
  File "test_placeholder.py", line 18, in <module>
    it = dset.make_one_shot_iterator()
  File "<...>/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 205, in make_one_shot_iterator
    six.reraise(ValueError, err)
  File "<...>/site-packages/six.py", line 692, in reraise
    raise value.with_traceback(tb)
ValueError: Cannot capture a placeholder (name:Placeholder, 
    type:Placeholder) by value.

测试：

import tensorflow as tf
import numpy as np

buf_size = 50
batch_size = 10
n_rows = 117

a = np.random.choice(7, size=n_rows)
b = np.random.uniform(0, 1, size=(n_rows, 4))

a_ph = tf.placeholder(a.dtype, a.shape)
b_ph = tf.placeholder(b.dtype, b.shape)

with tf.Session() as sess:
    dset = tf.data.Dataset.from_tensor_slices((a_ph, b_ph))
    dset = dset.shuffle(buf_size).batch(batch_size).repeat()
    feed_dict = {a_ph: a, b_ph: b}
    it = dset.make_one_shot_iterator()

    n_batches = len(a) // batch_size
    sess.run(it.initializer, feed_dict=feed_dict)
    for i in range(n_batches):
        a_chunk, b_chunk = it.get_next()
        print(a_chunk, b_chunk)

什么地方出了错？

谢谢。

答案

检查出importing data指南

“一个单次的迭代器是迭代器的最简单的形式，只支持迭代一次通过的数据集，而无需显式初始化。单次迭代处理几乎所有的情况下，现有的基于队列的输入管道支撑，但他们不支持参数“。

这是您的错误的原因，与占位符任何参数没有被这个特殊的迭代器的支持。我们可以利用make_initializable_iterator代替。

下面是与修改，你正在寻找的结果你的代码。

buf_size = 50
batch_size = 10
n_rows = 117

a = np.random.choice(7, size=n_rows)
b = np.random.uniform(0, 1, size=(n_rows, 4))

a_ph = tf.placeholder(a.dtype, a.shape)
b_ph = tf.placeholder(b.dtype, b.shape)

with tf.Session() as sess:
    dset = tf.data.Dataset.from_tensor_slices((a_ph, b_ph))
    dset = dset.shuffle(buf_size).batch(batch_size).repeat()
    feed_dict = {a_ph: a, b_ph: b}
    it = dset.make_initializable_iterator()

    n_batches = len(a) // batch_size
    sess.run(it.initializer, feed_dict=feed_dict)
    for i in range(n_batches):
        a_chunk, b_chunk = it.get_next()
        print(a_chunk, b_chunk)

结果：

Tensor("IteratorGetNext:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_1:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_1:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_2:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_2:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_3:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_3:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_4:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_4:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_5:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_5:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_6:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_6:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_7:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_7:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_8:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_8:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_9:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_9:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_10:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_10:1", shape=(?, 4), dtype=float64)