Re: Cache relation sizes?
Thomas Munro <thomas.munro@gmail.com>
From: Thomas Munro <thomas.munro@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>, "Jamison, Kirk" <k.jamison@jp.fujitsu.com>, "Tsunakawa, Takayuki" <tsunakawa.takay@jp.fujitsu.com>, "Ideriha, Takeshi" <ideriha.takeshi@jp.fujitsu.com>, pgsql-hackers <pgsql-hackers@postgresql.org>,
David Rowley <dgrowleyml@gmail.com>
Date: 2020-07-31T21:56:14Z
Lists: pgsql-hackers
Attachments
- 0001-WIP-Cache-smgrnblocks-in-more-cases.patch (text/x-patch) patch 0001
On Fri, Jul 31, 2020 at 2:36 PM Thomas Munro <thomas.munro@gmail.com> wrote: > There's still the matter of crazy numbers of lseeks in regular > backends; looking at all processes while running the above test, I get > 1,469,060 (9.18 per pgbench transaction) without -M prepared, and > 193,722 with -M prepared (1.21 per pgbench transaction). Fixing that > with this approach will require bullet-proof shared invalidation, but > I think it's doable, in later work. I couldn't help hacking on this a bit. Perhaps instead of bullet-proof general shared invalidation, we should have a way for localised bits of code to state that they are ok with a "relaxed" value. Then they should explain the theory for why that is safe in each case based on arguments about memory barrier pairs, but leave all other callers making the system call so that we don't torpedo the whole project by making it too hard. For now, the main cases I'm interested in are the ones that show up all the time as the dominant system call in various workloads: (1) Sequential scan relation-size probe. This should be OK with a relaxed value. You can't miss the invalidation for a truncation, because the read barrier in your lock acquisition on the relation pairs with the write barrier in the exclusive lock release of any session that truncated, and you can't miss relation any relation extension that your snapshot can see, because the release of the extension lock pairs with the lock involved in snapshot acquisition. (2) Planner relation-size probe, which should be OK with a relaxed value. Similar arguments give you a fresh enough view, I think. Or maybe there is a theory along these lines that already covers every use of smgrnblocks(), so a separate mode isn't require... I don't know! The attached sketch-quality patch shows some of the required elements working, though I ran out of steam trying to figure out how to thread this thing through the right API layers so for now it always asks for a relaxed value in table_block_relation_size().
Commits
-
Optimize DropRelFileNodesAllBuffers() for recovery.
- bea449c635c0 14.0 cited
-
Cache smgrnblocks() results in recovery.
- c5315f4f4484 14.0 landed
-
Use pg_pread() and pg_pwrite() for data files and WAL.
- c24dcd0cfd94 12.0 cited
-
Add a check to prevent overwriting valid data if smgrnblocks() gives a
- ffae5cc5a602 8.2.0 cited