Use Python "Limited API" in PL/Python
Peter Eisentraut <peter@eisentraut.org>
From: Peter Eisentraut <peter@eisentraut.org>
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2024-12-02T08:51:17Z
Lists: pgsql-hackers
Attachments
- 0001-Remove-obsolete-Python-version-check.patch (text/plain) patch 0001
- 0002-Use-Python-Limited-API-in-PL-Python.patch (text/plain) patch 0002
This patch changes PL/Python to use the Python "limited API". This API has stronger ABI stability guarantees.[0] This means, you can build PL/Python against any Python 3.x version and use any other Python 3.x version at run time. This is especially useful for binary packages where the operating system does not come with a fixed suitable version of Python. For example, Postgres.app (for macOS) would prefer to link against the Python version supplied by python.org (Python.app). But that has a 3.x version that changes over time. So instead they bundle a Python version inside Postgres.app. The Windows installer used to also bundle Python but as of PG17 you have to get it yourself, but you have to get a very specific version [1], which is unsatisfactory. This patch fixes that: You can use any Python version independent of what PL/Python was built against. (There is a mechanism to say "at least 3.N", but for this patch, we don't need that, we can stick with the current minimum of 3.2.) (I have only tested the macOS side of this, not the Windows side. In fact, the patch currently doesn't build on Windows on CI. I haven't figured out why.) For Linux-style packaging, I don't think this would have any benefit for users right now, since the OS comes with a Python installation and all the packages are built against that. But it could potentially be helpful for packagers. For example, on Debian, this could detach the postgresql packages from python version transitions. But AFAICT, the Python packaging layout is not prepared for that. (There are only libpython3.x.so libraries, no libpython3.so that one would have to link against.) Finally, I think this patch is part of a path toward making PL/Python thread-safe. I don't think the patch by itself changes anything, but if you read through [2], using heap types is part of the things mentioned there. [0]: https://docs.python.org/3/c-api/stable.html [1]: https://github.com/EnterpriseDB/edb-installers/blob/REL-17/server/resources/installation-notes.html#L34-L36 [2]: https://docs.python.org/3/howto/isolating-extensions.html
Commits
-
Enable Python Limited API for PL/Python on MSVC
- 2bc60f86219b 19 (unreleased) landed
-
Activate Python "Limited API" in PL/Python
- 0793ab810038 18.0 landed
-
Prepare for Python "Limited API" in PL/Python
- 72a3d0462b9a 18.0 landed
- c47e8df815c1 18.0 landed
-
Remove obsolete Python version check
- 32c393f9f1f1 18.0 landed