Re: Asynchronous and "direct" IO support for PostgreSQL.
Aleksander Alekseev <aleksander@timescale.com>
From: Aleksander Alekseev <aleksander@timescale.com>
To: Andres Freund <andres@anarazel.de>,
pgsql-hackers <pgsql-hackers@postgresql.org>
Cc: Julien Rouhaud <rjuju123@gmail.com>,
Thomas Munro <thomas.munro@gmail.com>, David Rowley <dgrowleyml@gmail.com>,
Melanie Plageman <melanieplageman@gmail.com>
Date: 2022-05-10T14:01:24Z
Lists: pgsql-hackers
Hi Andres, > > The code is at > > https://github.com/anarazel/postgres/tree/aio > > Just FYI the cfbot says that this version of the patchset doesn't > apply anymore, and it seems that your branch was only rebased to > 43c1c4f (Sept. 21th) which doesn't rebase cleanly: After watching your recent talk "IO in PostgreSQL: Past, Present, Future" [1] I decided to invest some of my time into this patchset. It looks like at very least it could use a reviewer, or maybe two :) Unfortunately, it's a bit difficult to work with the patchset at the moment. Any chance we may expect a rebased version for the July CF? > Comments? Questions? Personally, I'm very enthusiastic about this patchset. However, a set of 39 patches seems to be unrealistic to test and/or review and/or keep up to date. The 64 bit XIDs patchset [2] is much less complicated, but still it got the feedback that it should be splitted to more patches and CF entries. Any chance we could decompose this effort? For instance, I doubt that we need all the backends in the first implementation. The fallback "worker" one, and io_uring one will suffice. Other backends can be added as separate features. Considering that in any case the "worker" backend shouldn't cause any significant performance degradation, maybe we could start even without io_uring. BTW, do we need Posix AIO at all, given your feedback on this API? Also, what if we migrate to AIO/DIO one part of the system at a time? As I understood from your talk, sequential scans will benefit most from AIO/DIO. Will it be possible to improve them first, while part of the system will continue using buffered IO? [1]: https://www.youtube.com/watch?v=3Oj7fBAqVTw [2]: https://commitfest.postgresql.org/38/3594/ -- Best regards, Aleksander Alekseev
Commits
-
aio: Add README.md explaining higher level design
- fdd146a8ef2b 18.0 landed
-
bufmgr: Use AIO in StartReadBuffers()
- 12ce89fd0708 18.0 landed
-
bufmgr: Implement AIO read support
- 047cba7fa0f8 18.0 landed
-
aio: Implement support for reads in smgr/md/fd
- 50cb7505b301 18.0 landed
-
aio: Add io_method=io_uring
- c325a7633fcb 18.0 landed
-
aio: Add io_method=worker
- 247ce06b883d 18.0 landed
-
aio: Infrastructure for io_method=worker
- 55b454d0e140 18.0 landed
-
aio: Add core asynchronous I/O infrastructure
- da7226993fd4 18.0 landed