Horton

Re: Refactoring the checkpointer's fsync request queue

Shawn Debnath <sdn@amazon.com>

View thread View on postgr.es
From: Shawn Debnath <sdn@amazon.com>
To: Thomas Munro <thomas.munro@enterprisedb.com>
Cc: Dmitry Dolgov <9erthalion6@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2019-02-13T02:58:43Z
Lists: pgsql-hackers

Attachments

On Wed, Jan 30, 2019 at 09:59:38PM -0800, Shawn Debnath wrote:

> I wonder if it might be better to introduce two different functions 
> catering to the two different use cases for forcing an immediate sync:
> 
> - sync a relation
>     smgrimmedsyncrel(SMgrRelation, ForkNumber)
> - sync a specific segment
>     smgrimmedsyncseg(SMgrRelation, ForkNumber, SegmentNumber)
> 
> This will avoid having to specify InvalidSegmentNumber for majority of 
> the callers today.

I have gone ahead and rebased the refactor patch so it could cleanly 
apply on heapam.c, see patch v7.

I am also attaching a patch (v8) that implements smgrimmedsyncrel() and 
smgrimmedsyncseg() as I mentioned in the previous email. It avoids 
callers to pass in InvalidSegmentNumber when they just want to sync the 
whole relation. As a side effect, I was able to get rid of some extra 
checkpointer.h includes.  

--
Shawn Debnath
Amazon Web Services (AWS)

Commits

  1. Fix bugs in mdsyncfiletag().

  2. Refactor the fsync queue for wider use.

  3. Don't forget about failed fsync() requests.

  4. PANIC on fsync() failure.

  5. Move LockClauseStrength, LockWaitPolicy into new file nodes/lockoptions.h.

  6. Add new file for checkpointer.c

  7. Split work of bgwriter between 2 processes: bgwriter and checkpointer.

  8. Install infrastructure for shared-memory free space map. Doesn't actually