Pipe related future not retrieved if there's a BrokenPipeError

[nullstd]

Bug report

When running the following code, I ran into some unresolved future error message as shown below:

#!/usr/bin/env python

import sys
import os
import asyncio

async def main_task():

    print(sys.version)

    xxx = "x" * 196428
    incoming_original_data = xxx.encode()

    print(f"stdin len: {len(incoming_original_data)}")

    proc = await asyncio.create_subprocess_shell(
        f"xxd -l 1",
        stdin=asyncio.subprocess.PIPE,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE)
    
    await proc.communicate(input = incoming_original_data)

    print("subprocess ended")

def main():

    opts = asyncio.run(main_task())

    print(f"main END")

    return opts


if __name__ == '__main__':

    main()

Output

3.11.3 (main, Apr  7 2023, 19:25:52) [Clang 14.0.0 (clang-1400.0.29.202)]
stdin len: 196428
subprocess ended
main END
Future exception was never retrieved
future: <Future finished exception=BrokenPipeError(32, 'Broken pipe')>
Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/subprocess.py", line 152, in _feed_stdin
    await self.stdin.drain()
  File "/usr/local/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/streams.py", line 378, in drain
    await self._protocol._drain_helper()
  File "/usr/local/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/streams.py", line 173, in _drain_helper
    await waiter
  File "/usr/local/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/unix_events.py", line 709, in _write_ready
    n = os.write(self._fileno, self._buffer)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
BrokenPipeError: [Errno 32] Broken pipe

Because the child process exited so early, a BrokenPipeError was raised, and the event loop tries to disconnect relevant internal pipes, but one future _stdin_closed ( in subprocess.py ) is not properly handled. It's introduced in #13098

I also tested on Python 3.7.17 there's no such error message.

Your environment

• CPython versions tested on: Python 3.11.3, 3.7.17
• Operating system and architecture: macOS Ventura 13.3

Some links I found helpful:

https://stackoverflow.com/questions/14207708/ioerror-errno-32-broken-pipe-when-piping-prog-py-othercmd

https://stackoverflow.com/questions/23688492/oserror-errno-22-invalid-argument-in-subprocess#28020096

Could you help take a look? @asvetlov

cc @1st1

[duaneg]

This can't be reproduced on current head. Bisecting shows this is because #104340 suppressed the error. Although the details are a bit different, I think the same reasoning applies in both issues, and the current behaviour is good: the exception is there for users who care whether it was raised (e.g. add await proc.stdin.wait_closed() after await proc.communicate...), but can be ignored by those who don't.

I think this ticket can be closed.