Repository navigation
PEP 399: exception to remove io module Python implementation _pyio #372
Description
CPython currently contains two I/O stacks: _io written in C and _pyio written in Python. I would like a PEP 399 exception to remove _pyio.
- There is no flag to get the
_pyioimplementation as the I/O stack used outside of tests / private import. - The
_pyioimplementation depends on_io. To use it without supporting C the extensions requires patching, (_pyioimportsio,ioimports_iounconditionaly). _pyiouses distinct algorithms from_io. It has significantly diverged over time and getting to matching features takes work (ex. Fix_pyio.BytesIOGC, Match exception in_pyio.FileIO.readall, test_pyio.BytesIO, Match exceptions in open(), make the same copies in .readall(), ...)
Both are currently run in CI by some I/O specific testing infrastructure. _pyio takes significantly longer to execute the same tests, especially on older/smaller machines (ex. Raspberry Pi). I've worked to optimize it but getting faster takes significant core work.
The features needed for fast Python native I/O stacks are well represented by asyncio. In improving _pyio performance asyncio asyncio_tcp got over 10% faster. I think asyncio is representative of in-python I/O performance and has actual users.
I did an audit of open source projects and while some used to use _pyio none use it anymore. PyPy uses the native _io stack.
Activity
savannahostrowski commentedon Nov 6, 2026
Thanks, Cody! The SC talked this over. Could you start a thread on Discourse to share what you found and see if anyone still relies on _pyio? We’d like to get some wider feedback before deciding.
cmaloney commentedon Nov 9, 2026
Working on a discuss post. I'm concerned more visibility will result in more voices to keep without changing the code state or maintenance of
_pyio. Is it okay if I include some framing around both known arguments and maintenace burden it creates?In particular:
_iois currently PEP 387 - Backwards Compatibility Policy maintained for fixes and features._pyiois partially maintained and has meaningful deviations from_io. Adding_ionibblerI need to match_io. PEP 399 applied directly would require a_pyionibblerat the same time. Going from one fully maintained implementation to four with bugfix obligations is a significant burden I want to make explicit.a. By core developers: making StringIO threadsafe, making TextIOWrapper faster.
b. By new contributors: Figuring out the right behavior is hard and keeping the multiple implementations in sync while iterating is hard. Adding BytesIO.peek took "~180 comments and 54 commits for a ~200 line change" to land and still needed small fixup
_pyio.BytesIO.peek()returns bytearray instead of bytes cpython#155389. Another: sys.stdin.read() throws a TypeError when stdin is set to be non-blocking cpython#109523.The cases I've encountered for keeping so far:
_io._pyiois up (https://github.com/python/cpython/commits/main/Lib/_pyio.py).I can definitely include paths like reintroducing
_pyioafter_ionibbleris done using the new model. Another option is leaving_pyioas it is during the rework. I personally found it useful to prototype Nibbler in python and have done so both by hand and with AI. Rebuilding the C design in Python I suspect will provide significant differential testing value and uncover nibbler bugs that should be fixed :).sobolevn commentedon Nov 9, 2026
Can we separate
_pyiointo the PyPI module for people who need it? This would solve the potential problem.