Horton

Re: Refactoring the checkpointer's fsync request queue

Thomas Munro <thomas.munro@gmail.com>

View thread View on postgr.es
From: Thomas Munro <thomas.munro@gmail.com>
To: Shawn Debnath <sdn@amazon.com>
Cc: Robert Haas <robertmhaas@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Andres Freund <andres@anarazel.de>, Dmitry Dolgov <9erthalion6@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2019-04-03T12:08:31Z
Lists: pgsql-hackers

Attachments

On Tue, Apr 2, 2019 at 11:09 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> I'm going to do some more testing and tidying tomorrow (for example I
> think the segment.h header is silly and I'd like to remove that), and
> commit this.

As a sanity check on the programming interface this thing gives you, I
tried teaching the SLRUs to use the fsync queue.  I finished up making
a few small improvements, but the main thing I learned is that
"handler" needs to be part of the hash table key.  I suppose the
discriminator could even be inside FileTag itself, but I chose to keep
it separate and introduce a new struct to hold hander enum + FileTag
in the hash table.

The 0001 patch is what I'm going to commit soon.  I don't know why
Shawn measured a performance change -- a bug in an earlier version? --
I can't see it, but I'm planning to look into that a bit more first.
I've attached the 0002 SLRU patch for interest, but I'm not planning
to commit that one.

-- 
Thomas Munro
https://enterprisedb.com

Commits

  1. Fix bugs in mdsyncfiletag().

  2. Refactor the fsync queue for wider use.

  3. Don't forget about failed fsync() requests.

  4. PANIC on fsync() failure.

  5. Move LockClauseStrength, LockWaitPolicy into new file nodes/lockoptions.h.

  6. Add new file for checkpointer.c

  7. Split work of bgwriter between 2 processes: bgwriter and checkpointer.

  8. Install infrastructure for shared-memory free space map. Doesn't actually