Re: 回复:Re: Cache relation sizes?
Thomas Munro <thomas.munro@gmail.com>
From: Thomas Munro <thomas.munro@gmail.com>
To: David Steele <david@pgmasters.net>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Konstantin Knizhnik <k.knizhnik@postgrespro.ru>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, 陈佳昕(步真) <buzhen.cjx@alibaba-inc.com>
Date: 2021-03-11T08:34:54Z
Lists: pgsql-hackers
Attachments
- v4-0001-WIP-Track-relation-sizes-in-shared-memory.patch (text/x-patch) patch v4-0001
- v4-0002-WIP-Provide-a-lock-free-fast-path-for-smgrnblocks.patch (text/x-patch) patch v4-0002
- v4-0003-update-fifo-to-lru-to-sweep-a-valid-cache.patch (text/x-patch) patch v4-0003
On Thu, Mar 4, 2021 at 2:39 AM David Steele <david@pgmasters.net> wrote: > On 1/18/21 10:42 PM, 陈佳昕(步真) wrote: > > I want to share a patch with you, I change the replacement algorithm > > from fifo to a simple lru. > > What do you think of this change? Ok, if I'm reading this right, it changes the replacement algorithm from "clock" to "gclock". Gclock is like PostgreSQL's buffer pool, except that here the usage count is limited with a GUC, instead of the universal constant 5. Certainly gclock is better than clock. I used the simplest algorithm I could think of, deliberately trying not to go down the cache replacement algorithm rabbit hole. I think we should match the buffer pool, however good/bad that may be, and adding a usage count does seem to do that. I had some trouble applying the patch, it seems to be corrupted, and to apply on top of some other pach (looks like a struct moved from a .c to a .h?), but I managed to figure it out and make the tests pass, and I've attached a rebase. > Also, your patch set from [1] no longer applies (and of course this > latest patch is confusing the tester as well). Rebased. The main change was commit bea449c6 "Optimize DropRelFileNodesAllBuffers() for recovery.". With this rebase, that optimisation now works in online, which is cool (by "online" I mean not just in recovery). Other changes in this version: * Added spinlock backoff recommended by Andres in his review earlier. Erm, that's it. There are still many review comments from Buzhen and Andres to address. The main thing being: how can we clean these objects in the background? That question being unanswered, there is a 0% chance of committing this in PostgreSQL 14. So I will now move this to the next CF. Thanks for the reviews so far!
Commits
-
Optimize DropRelFileNodesAllBuffers() for recovery.
- bea449c635c0 14.0 cited
-
Cache smgrnblocks() results in recovery.
- c5315f4f4484 14.0 landed
-
Use pg_pread() and pg_pwrite() for data files and WAL.
- c24dcd0cfd94 12.0 cited
-
Add a check to prevent overwriting valid data if smgrnblocks() gives a
- ffae5cc5a602 8.2.0 cited