python – Tensorflow Dataset.from_generator失败并出现pyfunc异常

内容导读

互联网集市收集整理的这篇技术教程文章主要介绍了python – Tensorflow Dataset.from_generator失败并出现pyfunc异常，小编现在分享给大家，供广大互联网技能从业者学习和参考。文章包含4454字，纯文字阅读大概需要7分钟。

内容图文

我正在尝试使用tensorflow的每晚1.4,因为我需要Dataset.from_generator将一些可变长度的数据集放在一起.这个简单的代码(来自here的想法)：

import tensorflow as tf

Dataset = tf.contrib.data.Dataset
it2 = Dataset.range(5).make_one_shot_iterator()

def _dataset_generator():
    while True:
        try:
            try:
                get_next = it2.get_next()
                yield get_next
            except tf.errors.OutOfRangeError:
                continue
        except tf.errors.OutOfRangeError:
            return

# Dataset.from_generator need tensorflow > 1.3 !
das_dataset = Dataset.from_generator(_dataset_generator,
                                     output_types=(tf.float32, tf.float32))
das_dataset_it = das_dataset.make_one_shot_iterator()
with tf.Session() as sess:
    while True:
        print(sess.run(it2.get_next()))
        print(sess.run(das_dataset_it.get_next()))

失败了,相当神秘：

C:\Dropbox\_\PyCharmVirtual\TF-NIGHTLY\Scripts\python.exe C:/Users/MrD/.PyCharm2017.2/config/scratches/scratch_55.py
0
2017-10-01 12:51:39.773135: W C:\tf_jenkins\home\workspace\tf-nightly-windows\M\windows\PY\35\tensorflow\core\framework\op_kernel.cc:1192] Invalid argument: 0-th value returned by pyfunc_0 is int32, but expects int64
     [[Node: PyFunc = PyFunc[Tin=[], Tout=[DT_INT64], token="pyfunc_0"]()]]
Traceback (most recent call last):
  File "C:\Dropbox\_\PyCharmVirtual\TF-NIGHTLY\lib\site-packages\tensorflow\python\client\session.py", line 1323, in _do_call
    return fn(*args)
  File "C:\Dropbox\_\PyCharmVirtual\TF-NIGHTLY\lib\site-packages\tensorflow\python\client\session.py", line 1302, in _run_fn
    status, run_metadata)
  File "C:\_\Python35\lib\contextlib.py", line 66, in __exit__
    next(self.gen)
  File "C:\Dropbox\_\PyCharmVirtual\TF-NIGHTLY\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 467, in raise_exception_on_not_ok_status
    c_api.TF_GetCode(status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: 0-th value returned by pyfunc_0 is int32, but expects int64
     [[Node: PyFunc = PyFunc[Tin=[], Tout=[DT_INT64], token="pyfunc_0"]()]]
     [[Node: IteratorGetNext_1 = IteratorGetNext[output_shapes=[<unknown>, <unknown>], output_types=[DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](OneShotIterator_1)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/MrD/.PyCharm2017.2/config/scratches/scratch_55.py", line 24, in <module>
    print(sess.run(das_dataset_it.get_next()))
  File "C:\Dropbox\_\PyCharmVirtual\TF-NIGHTLY\lib\site-packages\tensorflow\python\client\session.py", line 889, in run
    run_metadata_ptr)
  File "C:\Dropbox\_\PyCharmVirtual\TF-NIGHTLY\lib\site-packages\tensorflow\python\client\session.py", line 1120, in _run
    feed_dict_tensor, options, run_metadata)
  File "C:\Dropbox\_\PyCharmVirtual\TF-NIGHTLY\lib\site-packages\tensorflow\python\client\session.py", line 1317, in _do_run
    options, run_metadata)
  File "C:\Dropbox\_\PyCharmVirtual\TF-NIGHTLY\lib\site-packages\tensorflow\python\client\session.py", line 1336, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: 0-th value returned by pyfunc_0 is int32, but expects int64
     [[Node: PyFunc = PyFunc[Tin=[], Tout=[DT_INT64], token="pyfunc_0"]()]]
     [[Node: IteratorGetNext_1 = IteratorGetNext[output_shapes=[<unknown>, <unknown>], output_types=[DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](OneShotIterator_1)]]

Process finished with exit code 1

注意发电机工作正常：

with tf.Session() as sess:
    for k in _dataset_generator():
        print(sess.run(k))

打印：

0
1
2
3
4
Traceback (most recent call last):
...
tensorflow.python.framework.errors_impl.OutOfRangeError: End of sequence
     [[Node: IteratorGetNext_5 = IteratorGetNext[output_shapes=[[]], output_types=[DT_INT64], _device="/job:localhost/replica:0/task:0/cpu:0"](OneShotIterator)]]

正如所料.

这是一个错误,一个缺失的功能还是我误解了一些东西？

解决方法:

Dataset.from_generator()方法旨在将非TensorFlow Python代码连接到tf.data输入管道.例如,您可以从生成器生成简单的Python对象(例如int和str对象),列表或NumPy数组,并将它们转换为TensorFlow值.

但是,在您的示例代码中,您将得到it.get_next()的结果,它是一个tf.Tensor对象.这不受支持.
 如果需要在不同的数据集中捕获迭代器,可以在虚拟数据集上使用Dataset.map(),如下所示：

import tensorflow as tf

Dataset = tf.contrib.data.Dataset
it2 = Dataset.range(5).make_one_shot_iterator()

das_dataset = Dataset.from_tensors(0).repeat().map(lambda _: it2.get_next())
das_dataset_it = das_dataset.make_one_shot_iterator()
with tf.Session() as sess:
    while True:
        print(sess.run(it2.get_next()))
        print(sess.run(das_dataset_it.get_next()))

内容总结

以上是互联网集市为您收集整理的python – Tensorflow Dataset.from_generator失败并出现pyfunc异常全部内容，希望文章能够帮你解决python – Tensorflow Dataset.from_generator失败并出现pyfunc异常所遇到的程序开发问题。 如果觉得互联网集市技术教程内容还不错，欢迎将互联网集市网站推荐给程序员好友。

内容备注

版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 gblab@vip.qq.com 举报，一经查实，本站将立刻删除。

内容手机端

---