Horton

Re: Handing off SLRU fsyncs to the checkpointer

Thomas Munro <thomas.munro@gmail.com>

View thread View on postgr.es
From: Thomas Munro <thomas.munro@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Jakub Wartak <Jakub.Wartak@tomtom.com>, "alvherre@2ndquadrant.com" <alvherre@2ndquadrant.com>, Robert Haas <robertmhaas@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2020-09-25T00:53:37Z
Lists: pgsql-hackers

Attachments

On Fri, Sep 25, 2020 at 12:05 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Thomas Munro <thomas.munro@gmail.com> writes:
> > Tom, do you have any thoughts on ShutdownCLOG() etc?
>
> Hm, if we cannot reach that without first completing a shutdown checkpoint,
> it does seem a little pointless.

Thanks for the sanity check.

> It'd likely be a good idea to add a comment to CheckPointCLOG et al
> explaining that we expect $what-exactly to fsync the data we are writing
> before the checkpoint is considered done.

Good point.  Done like this:

+       /*
+        * Write dirty CLOG pages to disk.  This may result in sync
requests queued
+        * for later handling by ProcessSyncRequests(), as part of the
checkpoint.
+        */
        TRACE_POSTGRESQL_CLOG_CHECKPOINT_START(true);
-       SimpleLruFlush(XactCtl, true);
+       SimpleLruWriteAll(XactCtl, true);
        TRACE_POSTGRESQL_CLOG_CHECKPOINT_DONE(true);

Here's a new version.  The final thing I'm contemplating before
pushing this is whether there may be hidden magical dependencies in
the order of operations in CheckPointGuts(), which I've changed
around.  Andres, any comments?

Commits

  1. Remove unused function prototypes.

  2. Defer flushing of SLRU files.

  3. Improve the vacuum error context phase information.

  4. Cache smgrnblocks() results in recovery.

  5. Refactor the fsync queue for wider use.

  6. Increase maximum number of clog buffers.

  7. Make the number of CLOG buffers adaptive, based on shared_buffers.

  8. Replace implementation of pg_log as a relation accessed through the