Horton

Re: Improving connection scalability: GetSnapshotData()

Bruce Momjian <bruce@momjian.us>

View thread View on postgr.es
From: Bruce Momjian <bruce@momjian.us>
To: Robert Haas <robertmhaas@gmail.com>
Cc: "Jonathan S. Katz" <jkatz@postgresql.org>, Alexander Korotkov <a.korotkov@postgrespro.ru>, Andres Freund <andres@anarazel.de>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Peter Geoghegan <pg@bowt.ie>, Alvaro Herrera <alvherre@2ndquadrant.com>, Thomas Munro <thomas.munro@gmail.com>, David Rowley <dgrowleyml@gmail.com>
Date: 2020-04-08T22:06:23Z
Lists: pgsql-hackers 
On Wed, Apr  8, 2020 at 09:44:16AM -0400, Robert Haas wrote:
> I don't know what the right thing to do is. I agree with everyone who
> says this is a very important problem, and I have the highest respect
> for Andres's technical ability. On the other hand, I have been around
> here long enough to know that deciding whether to allow late commits
> on the basis of how much we like the feature is a bad plan, because it
> takes into account only the upside of a commit, and ignores the
> possible downside risk. Typically, the commit is late because the
> feature was rushed to completion at the last minute, which can have an
> effect on quality. I can say, having read through the patches
> yesterday, that they don't suck, but I can't say that they're fully
> correct. That's not to say that we shouldn't decide to take them, but
> it is a concern to be taken seriously. We have made mistakes before in
> what we shipped that had serious implications for many users and for
> the project; we should all be wary of making more such mistakes. I am
> not trying to say that solving problems and making stuff better is NOT
> important, just that every coin has two sides.

If we don't commit this, where does this leave us with the
old_snapshot_threshold feature?  We remove it in back branches and have
no working version in PG 13?  That seems kind of bad.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

Commits

  1. Try to unbreak 021_row_visibility.pl on mingw.

  2. Fix and test snapshot behavior on standby.

  3. Fix race condition in snapshot caching when 2PC is used.

  4. snapshot scalability: cache snapshots using a xact completion counter.

  5. Fix use of wrong index in ComputeXidHorizons().

  6. Make vacuum a bit more verbose to debug BF failure.

  7. snapshot scalability: Introduce dense array of in-progress xids.

  8. snapshot scalability: Move PGXACT->vacuumFlags to ProcGlobal->vacuumFlags.

  9. snapshot scalability: Move subxact info to ProcGlobal, remove PGXACT.

  10. snapshot scalability: Move PGXACT->xmin back to PGPROC.

  11. snapshot scalability: Don't compute global horizons while building snapshots.

  12. BRIN: Handle concurrent desummarization properly

  13. Track latest completed xid as a FullTransactionId.

  14. Rename VariableCacheData.nextFullXid to nextXid.

  15. snapshot scalability: Move delayChkpt from PGXACT to PGPROC.

  16. Report progress of CREATE INDEX operations