v4-0005-Prototype-of-opportunistic-freezing.patch

From 919bd29902da15a5d43166dabf379cdc60d7dacb Mon Sep 17 00:00:00 2001
From: Peter Geoghegan <pg@bowt.ie>
Date: Mon, 13 Dec 2021 15:00:49 -0800
Subject: [PATCH v4 5/5] Prototype of opportunistic freezing.

Freeze whenever pruning modified the page, or whenever we see that we're
going to mark the page all-visible without also marking it all-frozen.

There has been plenty of discussion of opportunistic freezing in the
past.  It is generally considered important as a way of minimizing
repeated dirtying of heap pages (or the total volume of FPIs in the WAL
stream) over time.  While that goal is certainly very important, this
patch has another priority: making VACUUM advance relfrozenxid sooner
and more frequently.

The overall effect is that tables like pgbench's history table can be
vacuumed very frequently, and have most individual vacuum operations
generate 0 FPIs in WAL -- they will never need an aggressive VACUUM.
The old SKIP_PAGES_THRESHOLD heuristic is designed to make it more
likely that we'll be able to advance relfrozenxid, which works well when
combined with additions from earlier patches in the patch series, and
opportunistic freezing.

GUCs like vacuum_freeze_min_age never made much sense after the freeze
map work in PostgreSQL 9.6.  The default is 50 million transactions,
which current tends to result in our being unable to freeze tuples
before the page is marked all-visible (but not all-frozen).  This
creates a huge performance cliff later on, during the first aggressive
VACUUM.  And so an important goal of opportunistic freezing is to not
allow the system to get into too much "debt" from very old unfrozen
tuples.  That might actually be more important than minimizing the
absolute cost of freezing.

There is probably a small regression caused by opportunistic freezing
with workloads like pgbench, since we're freezing many more tuples than
we need to now -- while we do have fewer FPIs (even earlier on), that
may not be enough to make up for the increase in WAL records.  This
problem can be addressed in a later revision, when the general picture
for this patch (especially how it affects our ability to advance
relfrozenxid early) becomes clearer.
---
 src/include/access/heapam.h          |  1 +
 src/backend/access/heap/pruneheap.c  |  8 ++-
 src/backend/access/heap/vacuumlazy.c | 78 +++++++++++++++++++++++++---
 3 files changed, 79 insertions(+), 8 deletions(-)

diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 0eb5c36a2..5e1f24e5c 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -188,6 +188,7 @@ extern int	heap_page_prune(Relation relation, Buffer buffer,
 							struct GlobalVisState *vistest,
 							TransactionId old_snap_xmin,
 							TimestampTz old_snap_ts_ts,
+							bool *modified,
 							int	*nnewlpdead,
 							OffsetNumber *off_loc);
 extern void heap_page_prune_execute(Buffer buffer,
diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c
index 522a00af6..e95dea38d 100644
--- a/src/backend/access/heap/pruneheap.c
+++ b/src/backend/access/heap/pruneheap.c
@@ -182,11 +182,12 @@ heap_page_prune_opt(Relation relation, Buffer buffer)
 		 */
 		if (PageIsFull(page) || PageGetHeapFreeSpace(page) < minfree)
 		{
+			bool	modified;
 			int		ndeleted,
 					nnewlpdead;
 
 			ndeleted = heap_page_prune(relation, buffer, vistest, limited_xmin,
-									   limited_ts, &nnewlpdead, NULL);
+									   limited_ts, &modified, &nnewlpdead, NULL);
 
 			/*
 			 * Report the number of tuples reclaimed to pgstats.  This is
@@ -244,6 +245,7 @@ heap_page_prune(Relation relation, Buffer buffer,
 				GlobalVisState *vistest,
 				TransactionId old_snap_xmin,
 				TimestampTz old_snap_ts,
+				bool *modified,
 				int	*nnewlpdead,
 				OffsetNumber *off_loc)
 {
@@ -375,6 +377,8 @@ heap_page_prune(Relation relation, Buffer buffer,
 
 			PageSetLSN(BufferGetPage(buffer), recptr);
 		}
+
+		*modified = true;
 	}
 	else
 	{
@@ -387,12 +391,14 @@ heap_page_prune(Relation relation, Buffer buffer,
 		 * point in repeating the prune/defrag process until something else
 		 * happens to the page.
 		 */
+		*modified = false;
 		if (((PageHeader) page)->pd_prune_xid != prstate.new_prune_xid ||
 			PageIsFull(page))
 		{
 			((PageHeader) page)->pd_prune_xid = prstate.new_prune_xid;
 			PageClearFull(page);
 			MarkBufferDirtyHint(buffer, true);
+			*modified = true;
 		}
 	}
 
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index c6facc9eb..a710c6cf8 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -328,6 +328,7 @@ typedef struct LVRelState
 
 	/* VACUUM operation's cutoff for pruning */
 	TransactionId OldestXmin;
+	MultiXactId OldestMxact;
 	/* VACUUM operation's cutoff for freezing XIDs and MultiXactIds */
 	TransactionId FreezeLimit;
 	MultiXactId MultiXactCutoff;
@@ -529,11 +530,16 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
 
 	/*
 	 * Get cutoffs that determine which tuples we need to freeze during the
-	 * VACUUM operation.
+	 * VACUUM operation.  This includes information that is used during
+	 * opportunistic freezing, where the most aggressive possible cutoffs
+	 * (OldestXmin and OldestMxact) are used for some heap pages, based on
+	 * considerations about cost.
 	 *
 	 * Also determines if this is to be an aggressive VACUUM.  This will
 	 * eventually be required for any table where (for whatever reason) no
 	 * non-aggressive VACUUM ran to completion, and advanced relfrozenxid.
+	 * This used to be much more common, but we now work hard to advance
+	 * relfrozenxid in non-aggressive VACUUMs.
 	 */
 	aggressive = vacuum_set_xid_limits(rel,
 									   params->freeze_min_age,
@@ -603,6 +609,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
 
 	/* Set cutoffs for entire VACUUM */
 	vacrel->OldestXmin = OldestXmin;
+	vacrel->OldestMxact = OldestMxact;
 	vacrel->FreezeLimit = FreezeLimit;
 	vacrel->MultiXactCutoff = MultiXactCutoff;
 
@@ -1807,6 +1814,10 @@ lazy_scan_prune(LVRelState *vacrel,
 	xl_heap_freeze_tuple frozen[MaxHeapTuplesPerPage];
 	TransactionId NewRelfrozenxid;
 	MultiXactId NewRelminmxid;
+	bool		modified;
+	TransactionId FreezeLimit = vacrel->FreezeLimit;
+	MultiXactId MultiXactCutoff = vacrel->MultiXactCutoff;
+	bool		earlyfreezing = false;
 
 	Assert(BufferGetBlockNumber(buf) == blkno);
 
@@ -1833,8 +1844,19 @@ retry:
 	 * that were deleted from indexes.
 	 */
 	tuples_deleted = heap_page_prune(rel, buf, vistest,
-									 InvalidTransactionId, 0, &nnewlpdead,
-									 &vacrel->offnum);
+									 InvalidTransactionId, 0, &modified,
+									 &nnewlpdead, &vacrel->offnum);
+
+	/*
+	 * If page was modified during pruning, then perform early freezing
+	 * opportunistically
+	 */
+	if (!earlyfreezing && modified)
+	{
+		earlyfreezing = true;
+		FreezeLimit = vacrel->OldestXmin;
+		MultiXactCutoff = vacrel->OldestMxact;
+	}
 
 	/*
 	 * Now scan the page to collect LP_DEAD items and check for tuples
@@ -1889,7 +1911,7 @@ retry:
 		if (ItemIdIsDead(itemid))
 		{
 			deadoffsets[lpdead_items++] = offnum;
-			prunestate->all_visible = false;
+			/* Don't set all_visible to false just yet */
 			prunestate->has_lpdead_items = true;
 			continue;
 		}
@@ -2023,8 +2045,8 @@ retry:
 		if (heap_prepare_freeze_tuple(tuple.t_data,
 									  vacrel->relfrozenxid,
 									  vacrel->relminmxid,
-									  vacrel->FreezeLimit,
-									  vacrel->MultiXactCutoff,
+									  FreezeLimit,
+									  MultiXactCutoff,
 									  &frozen[nfrozen],
 									  &tuple_totally_frozen,
 									  &NewRelfrozenxid,
@@ -2044,6 +2066,48 @@ retry:
 
 	vacrel->offnum = InvalidOffsetNumber;
 
+	/*
+	 * If page is going to become all_visible (excluding any LP_DEAD items),
+	 * but won't also become all_frozen (either in ongoing first heap pass, or
+	 * in second heap pass after LP_DEAD items get set LP_UNUSED) then repeat
+	 * our pass over the heap, using more aggressive (opportunistic) freeze
+	 * limits.  This policy isn't guaranteed to be cheaper in the long run,
+	 * but it often is.  And it makes it far more likely that non-aggressive
+	 * VACUUMs will end up advancing relfrozenxid to a reasonably recent XID;
+	 * an XID that we opt to freeze won't hold back NewRelfrozenxid.
+	 *
+	 * We deliberately track all_visible in a way that excludes LP_DEAD items
+	 * here.  Our assumption is that any page that is "all_visible for tuples
+	 * with storage" will be safe to mark all_visible in the visibility map
+	 * during VACUUM's second heap pass, right after LP_DEAD items are set
+	 * LP_UNUSED.  Either way (with or without LP_DEAD items), our goal is to
+	 * ensure that a page that _would have_ been marked all_visible in the
+	 * visibility map gets marked all_frozen instead.
+	 */
+	if (!earlyfreezing && prunestate->all_visible && !prunestate->all_frozen)
+	{
+		/*
+		 * XXX Need to worry about leaking MultiXacts in FreezeMultiXactId()
+		 * now (via heap_prepare_freeze_tuple calls)?  That was already
+		 * possible, but presumably this makes it much more likely.
+		 *
+		 * On the other hand, that's only possible when we need to replace an
+		 * existing MultiXact with a new one.  Even then, we won't have
+		 * preallocated a new MultiXact (which we now risk leaking) if there
+		 * was only one remaining XID, and the XID is for an updater (we'll
+		 * only prepare to replace xmax with the XID directly).  So maybe it's
+		 * still a narrow enough problem to be ignored.
+		 */
+		earlyfreezing = true;
+		FreezeLimit = vacrel->OldestXmin;
+		MultiXactCutoff = vacrel->OldestMxact;
+		goto retry;
+	}
+
+	/* Time to define all_visible in a way that accounts for LP_DEAD items */
+	if (lpdead_items > 0)
+		prunestate->all_visible = false;
+
 	/*
 	 * We have now divided every item on the page into either an LP_DEAD item
 	 * that will need to be vacuumed in indexes later, or a LP_NORMAL tuple
@@ -2089,7 +2153,7 @@ retry:
 		{
 			XLogRecPtr	recptr;
 
-			recptr = log_heap_freeze(vacrel->rel, buf, vacrel->FreezeLimit,
+			recptr = log_heap_freeze(vacrel->rel, buf, FreezeLimit,
 									 frozen, nfrozen);
 			PageSetLSN(page, recptr);
 		}
-- 
2.30.2