Horton

Re: [Patch] Optimize dropping of relation buffers using dlist

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, "k.jamison@fujitsu.com" <k.jamison@fujitsu.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2020-08-07T04:03:13Z
Lists: pgsql-hackers 
Amit Kapila <amit.kapila16@gmail.com> writes:
> On Sat, Aug 1, 2020 at 1:53 AM Andres Freund <andres@anarazel.de> wrote:
>> We could also just use pg_class.relpages. It'll probably mostly be
>> accurate enough?

> Don't we need the accurate 'number of blocks' if we want to invalidate
> all the buffers? Basically, I think we need to perform BufTableLookup
> for all the blocks in the relation and then Invalidate all buffers.

Yeah, there is no room for "good enough" here.  If a dirty buffer remains
in the system, the checkpointer will eventually try to flush it, and fail
(because there's no file to write it to), and then checkpointing will be
stuck.  So we cannot afford to risk missing any buffers.

			regards, tom lane

Commits

  1. Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.

  2. Optimize DropRelFileNodesAllBuffers() for recovery.

  3. Optimize DropRelFileNodeBuffers() for recovery.

  4. Cache smgrnblocks() results in recovery.

  5. Add a check to prevent overwriting valid data if smgrnblocks() gives a