Horton

Re: Make relfile tombstone files conditional on WAL level

Thomas Munro <thomas.munro@gmail.com>

View thread View on postgr.es
From: Thomas Munro <thomas.munro@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Heikki Linnakangas <hlinnaka@iki.fi>, pgsql-hackers <pgsql-hackers@postgresql.org>, Simon Riggs <simon.riggs@enterprisedb.com>
Date: 2021-10-05T03:21:39Z
Lists: pgsql-hackers 
On Thu, Sep 30, 2021 at 11:32 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> I managed to produce a case where live data is written to an unlinked
> file and lost

I guess this must have been broken since release 9.2 moved checkpoints
out of here[1].  The connection between checkpoints, tombstone files
and file descriptor cache invalidation in auxiliary (non-sinval)
backends was not documented as far as I can see (or at least not
anywhere near the load-bearing parts).

How could it be fixed, simply and backpatchably?  If BgSyncBuffer()
did if-FirstCallSinceLastCheckpoint()-then-smgrcloseall() after
locking each individual buffer and before flushing, then I think it
might logically have the correct interlocking against relfilenode
wraparound, but that sounds a tad expensive :-(  I guess it could be
made cheaper by using atomics for the checkpoint counter instead of
spinlocks.  Better ideas?

[1] https://www.postgresql.org/message-id/flat/CA%2BU5nMLv2ah-HNHaQ%3D2rxhp_hDJ9jcf-LL2kW3sE4msfnUw9gA%40mail.gmail.com

Commits

  1. Fix typos in comments, code and documentation

  2. Increase width of RelFileNumbers from 32 bits to 56 bits.

  3. Move RelFileNumber declarations to common/relpath.h.

  4. Include RelFileLocator fields individually in BufferTag.

  5. Convert macros to static inline functions (buf_internals.h)

  6. Remove the restriction that the relmap must be 512 bytes.

  7. Change internal RelFileNode references to RelFileNumber or RelFileLocator.

  8. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  9. Fix DROP {DATABASE,TABLESPACE} on Windows.

  10. Restructure smgr API as per recent proposal. smgr no longer depends on