Python 和 asyncio：封闭的命名管道始终可供读取

Posted 2023-02-16

【中文标题】Python 和 asyncio：封闭的命名管道始终可供读取

【英文标题】：Python and asyncio: closed named pipe always available for reading

【发布时间】：2018-03-26 21:56:39

【问题描述】：

我正在编写一个脚本，该脚本通过命名管道从另一个软件中读取数据。我只想在可用时读取数据，并且我尝试使用add_reader from asyncio。

我注意到，在 Linux 上，我注册的阅读器在管道关闭后被连续调用。在 macOS 上，这不会发生。

这让我很困惑，因为在管道的写入端挂了之后，我不希望读取端可以读取，尤其是因为显然不可能没有数据。

这个脚本说明了这个问题：

#!/usr/bin/env python3
import os, asyncio, threading, time
NAMED_PIPE = 'write.pipe'

# Setup the named pipe
if os.path.exists(NAMED_PIPE):
    os.unlink(NAMED_PIPE)
os.mkfifo(NAMED_PIPE)

loop = asyncio.get_event_loop()

def simulate_write():
    # Open the pipe for writing and write something into it.
    # This simulates another process
    print('waiting for opening pipe for writing')
    with open(NAMED_PIPE, 'w') as write_stream:
        print('writing pipe opened')
        time.sleep(1)
        print('writing some data')
        print('<some data>', file=write_stream)
        time.sleep(1)
    print('exiting simulated write')


async def open_pipe_for_reading():
    print('waiting for opening pipe for reading')
    # This needs to run asynchronously because open will
    # not reuturn until on the other end, someone tries
    # to write
    return open(NAMED_PIPE)

count = 0
def read_data_block(fd):
    global count
    count += 1
    print('reading data', fd.read())
    if count > 10:
        print('reached maximum number of calls')
        loop.remove_reader(fd.fileno())

# Spawn a thread that will simulate writing
threading.Thread(target=simulate_write).start()
# Get the result of open_pipe_for_reading
stream = loop.run_until_complete(open_pipe_for_reading())
print('reading pipe opened')
# Schedule the reader
loop.add_reader(stream.fileno(), read_data_block, stream)
try:
    loop.run_forever()
except KeyboardInterrupt:
    pass
finally:
    print('closing stream')
    stream.close()
print('removing pipe')
os.unlink(NAMED_PIPE)

在 OSX 上，这是我观察到的行为：

waiting for opening pipe for writing
waiting for opening pipe for reading
reading pipe opened
writing pipe opened
writing some data
exiting simulated write
reading data <some data>

^Cclosing stream
removing pipe

在 Linux 上：

waiting for opening pipe for writing
waiting for opening pipe for reading
reading pipe opened
writing pipe opened
writing some data
exiting simulated write
reading data <some data>

reading data
reading data
reading data
reading data
reading data
reading data
reading data
reading data
reading data
reading data
reached maximum number of calls
^C<closing stream
removing pipe

那么，为什么没有数据的封闭管道可以读取？

另外，据我了解，add_reader 会在可以从 读取流并且 有一些数据要读取时触发； 这个解释正确吗？

---

Python 和操作系统版本：

Python 3.6.4 (MacPorts)、macOS High Sierra 10.13.3 (17D102) Python 3.6.1（手动编译）CentOS Linux release 7.4.1708（核心） Python 3.5.2（来自 repo）Linux Mint 18.2 Sonya

【参考方案1】：

在 python 中读取空数据是套接字/管道关闭的标志。

data = fd.read()
if not data:
    return

另外请将管道切换到非阻塞模式：

os.set_blocking(fd, False)

【讨论】：

实际上，如果我将其设置为非阻塞，则每当我在没有数据可用时读取时，我都会得到空数据，尽管管道仍处于打开状态（尽管在 Linux 上可以检测何时管道已关闭并重新安排阅读器）。不过，这并不能解释 add_reader 背后的基本原理。