Horton

Re: Improving connection scalability: GetSnapshotData()

Michael Paquier <michael@paquier.xyz>

View thread View on postgr.es
From: Michael Paquier <michael@paquier.xyz>
To: Andres Freund <andres@anarazel.de>
Cc: "Jonathan S. Katz" <jkatz@postgresql.org>, Alexander Korotkov <a.korotkov@postgrespro.ru>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Peter Geoghegan <pg@bowt.ie>, Alvaro Herrera <alvherre@2ndquadrant.com>, Bruce Momjian <bruce@momjian.us>, Thomas Munro <thomas.munro@gmail.com>, David Rowley <dgrowleyml@gmail.com>
Date: 2020-04-09T01:22:04Z
Lists: pgsql-hackers 
On Wed, Apr 08, 2020 at 03:17:41PM -0700, Andres Freund wrote:
> On 2020-04-08 09:26:42 -0400, Jonathan S. Katz wrote:
>> Lastly, with the ongoing world events, perhaps time that could have been
>> dedicated to this and other patches likely affected their completion. I
>> know most things in my life take way longer than they used to (e.g.
>> taking out the trash/recycles has gone from a 15s to 240s routine). The
>> same could be said about other patches as well, but this one has a far
>> greater impact (a double-edged sword, of course) given it's a feature
>> that everyone uses in PostgreSQL ;)
> 
> I'm obviously not alone in that, so I agree that it's not an argument
> pro/con anything.
> 
> But this definitely is the case for me. Leaving aside the general dread,
> not having a quiet home-office, nor good exercise, is definitely not
> helping.

Another factor to be careful of is that by committing a new feature in
a release cycle, you actually need to think about the extra amount of
resources you may need to address comments and issues about it in time
during the beta/stability period, and that more care is likely needed
if you commit something at the end of the cycle.  On top of that,
currently, that's a bit hard to plan one or two weeks ahead if help is
needed to stabilize something you worked on.  I am pretty sure that
we'll be able to sort things out with a collective effort though.
--
Michael

Commits

  1. Try to unbreak 021_row_visibility.pl on mingw.

  2. Fix and test snapshot behavior on standby.

  3. Fix race condition in snapshot caching when 2PC is used.

  4. snapshot scalability: cache snapshots using a xact completion counter.

  5. Fix use of wrong index in ComputeXidHorizons().

  6. Make vacuum a bit more verbose to debug BF failure.

  7. snapshot scalability: Introduce dense array of in-progress xids.

  8. snapshot scalability: Move PGXACT->vacuumFlags to ProcGlobal->vacuumFlags.

  9. snapshot scalability: Move subxact info to ProcGlobal, remove PGXACT.

  10. snapshot scalability: Move PGXACT->xmin back to PGPROC.

  11. snapshot scalability: Don't compute global horizons while building snapshots.

  12. BRIN: Handle concurrent desummarization properly

  13. Track latest completed xid as a FullTransactionId.

  14. Rename VariableCacheData.nextFullXid to nextXid.

  15. snapshot scalability: Move delayChkpt from PGXACT to PGPROC.

  16. Report progress of CREATE INDEX operations