Re: 回复:Re: Cache relation sizes?

Thomas Munro <thomas.munro@gmail.com>

From: Thomas Munro <thomas.munro@gmail.com>
To: David Steele <david@pgmasters.net>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Konstantin Knizhnik <k.knizhnik@postgrespro.ru>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, 陈佳昕(步真) <buzhen.cjx@alibaba-inc.com>
Date: 2021-03-11T08:34:54Z
Lists: pgsql-hackers

Attachments

On Thu, Mar 4, 2021 at 2:39 AM David Steele <david@pgmasters.net> wrote:
> On 1/18/21 10:42 PM, 陈佳昕(步真) wrote:
> > I want to share a patch with you, I change the replacement algorithm
> > from fifo to a simple lru.
>
> What do you think of this change?

Ok, if I'm reading this right, it changes the replacement algorithm
from "clock" to "gclock".  Gclock is like PostgreSQL's buffer pool,
except that here the usage count is limited with a GUC, instead of the
universal constant 5.  Certainly gclock is better than clock.  I used
the simplest algorithm I could think of, deliberately trying not to go
down the cache replacement algorithm rabbit hole.  I think we should
match the buffer pool, however good/bad that may be, and adding a
usage count does seem to do that.

I had some trouble applying the patch, it seems to be corrupted, and
to apply on top of some other pach (looks like a struct moved from a
.c to a .h?), but I managed to figure it out and make the tests pass,
and I've attached a rebase.

> Also, your patch set from [1] no longer applies (and of course this
> latest patch is confusing the tester as well).

Rebased.  The main change was commit bea449c6 "Optimize
DropRelFileNodesAllBuffers() for recovery.".  With this rebase, that
optimisation now works in online, which is cool (by "online" I mean
not just in recovery).

Other changes in this version:

* Added spinlock backoff recommended by Andres in his review earlier.

Erm, that's it.  There are still many review comments from Buzhen and
Andres to address.  The main thing being: how can we clean these
objects in the background?  That question being unanswered, there is a
0% chance of committing this in PostgreSQL 14.  So I will now move
this to the next CF.

Thanks for the reviews so far!

Commits

  1. Optimize DropRelFileNodesAllBuffers() for recovery.

  2. Cache smgrnblocks() results in recovery.

  3. Use pg_pread() and pg_pwrite() for data files and WAL.

  4. Add a check to prevent overwriting valid data if smgrnblocks() gives a