Horton

Re: Refactoring the checkpointer's fsync request queue

Shawn Debnath <sdn@amazon.com>

View thread View on postgr.es
From: Shawn Debnath <sdn@amazon.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Dmitry Dolgov <9erthalion6@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2019-04-04T05:41:07Z
Lists: pgsql-hackers 
On Thu, Apr 04, 2019 at 05:39:14PM +1300, Thomas Munro wrote:
> On Thu, Apr 4, 2019 at 5:36 PM Andres Freund <andres@anarazel.de> wrote:
> > On 2019-04-03 21:19:45 -0700, Shawn Debnath wrote:
> > > +typedef struct FileTag
> > > +{
> > > +     int16           handler;                /* SyncRequstHandler value, saving space */
> > > +     int16           forknum;                /* ForkNumber, saving space */
> > > +     RelFileNode rnode;
> > > +     BlockNumber segno;
> > > +} FileTag;
> >
> > Seems odd to me to use BlockNumber for segno.
> 
> That is a tradition in md.c code.  I had a new typedef SegmentNumber
> in all sync.{c,h} stuff in an earlier version, but had trouble
> figuring out where to define it...

Thomas, this is why I had defined segment.h with the contents below :-)

+++ b/src/include/storage/segment.h
[...]
+/*
+ * Segment Number:
+ *
+ * Each relation and its forks are divided into segments. This
+ * definition formalizes the definition of the segment number.
+ */
+typedef uint32 SegmentNumber;
+
+#define InvalidSegmentNumber ((SegmentNumber) 0xFFFFFFFF)

My last iteration, v12, patch had it. See [1] for comments on removal of 
segment.h.


[1] 
https://www.postgresql.org/message-id/20190403214423.GA45392%40f01898859afd.ant.amazon.com

-- 
Shawn Debnath
Amazon Web Services (AWS)

Commits

  1. Fix bugs in mdsyncfiletag().

  2. Refactor the fsync queue for wider use.

  3. Don't forget about failed fsync() requests.

  4. PANIC on fsync() failure.

  5. Move LockClauseStrength, LockWaitPolicy into new file nodes/lockoptions.h.

  6. Add new file for checkpointer.c

  7. Split work of bgwriter between 2 processes: bgwriter and checkpointer.

  8. Install infrastructure for shared-memory free space map. Doesn't actually