Repository navigation
Extension isolation and subinterpreters support #226
Description
We are integrating Python as an embedded interpreter in a C++ application to provide a plugin interface for end users. Our codebase is transitioning to a multi-threaded architecture, where each independent unit of work utilizes its own subinterpreter with a separate GIL. This functionality became available in Python 3.12.
We have modified the bundled version of python-rapidjson to ensure compatibility with our setup. The modified version can be found here: https://github.com/yandex/yatool/blob/main/devtools/ymake/contrib/python-rapidjson/rapidjson.cpp
I want to refactor and upstream the subinterpreters compatibility changes in several PRs, ensuring clean and maintainable code.
Are you interested in this feature?
Note: I'm aware that some projects consider subinterpreters to be not fully mature for production use. Therefore, I would like to assess the community’s interest before proceeding with the development.
Activity
lelit commentedon Aug 30, 2025
This is similar to issue #225: I bet that as new core Python features become mainstream, there will an increasing interest in having them supported by 3rd party packages, so yes, I think it is sensible, at least when the required changes are not (too) disruptive, well tested and the performance impact is taken in consideration.
OTOH, for the foreseeable future I won't have a direct motivation to use those features myself, so the best thing would be if an actual user could lead the transition, and you seem a good candidate here 😉
Thanks in advance!
VestniK commentedon Sep 3, 2025
I've sent the first PR to get basic feedback on coding style or get some other important notices on what fits the module code base and what does not.
The roadmap looks like:
PyObjects and other mutable global state into a single struct.lelit commentedon Sep 14, 2025
Sorry for the lag, enjoyed a short vacation 😉
Thank you, I will inspect your contribution as soon as possible.
lelit commentedon Aug 7, 2026
... and then completely forgot about this, sigh! Sorry, I will try to re-focus.