Horton

Re: Improving connection scalability: GetSnapshotData()

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Alexander Korotkov <a.korotkov@postgrespro.ru>
Cc: Peter Geoghegan <pg@bowt.ie>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2020-04-06T23:52:28Z
Lists: pgsql-hackers 
Hi,

On 2020-04-06 06:39:59 -0700, Andres Freund wrote:
> > 3) Plain pgbench read-write (you already did it for sure)
> 
> -s 100 -M prepared -T 700
> 
> autovacuum=off, fsync on:
> clients     tps master      tps pgxact
> 1           474             479
> 16          4356            4476
> 40          8591            9309
> 198         20045           20261
> 1024        17986           18545
> 
> autovacuum=off, fsync off:
> clients     tps master      tps pgxact
> 1           7828            7719
> 16          49069           50482
> 40          68241           73081
> 198         73464           77801
> 1024        25621           28410
> 
> I chose autovacuum off because otherwise the results vary much more
> widely, and autovacuum isn't really needed for the workload.

> These numbers seem pretty decent to me. The regressions seem mostly
> within noise. The one possible exception to that is plain pgbench
> read/write with fsync=off and only a single session. I'll run more
> benchmarks around that tomorrow (but now it's 6am :().

The "one possible exception" turned out to be a "real" regression, but
one that was dead easy to fix: It was an DEBUG1 elog I had left in. The
overhead seems to solely have been the increased code size + overhead of
errstart().  After that there's no difference in the single client case
anymore (I'd not expect a benefit).

Greetings,

Andres Freund

Commits

  1. Try to unbreak 021_row_visibility.pl on mingw.

  2. Fix and test snapshot behavior on standby.

  3. Fix race condition in snapshot caching when 2PC is used.

  4. snapshot scalability: cache snapshots using a xact completion counter.

  5. Fix use of wrong index in ComputeXidHorizons().

  6. Make vacuum a bit more verbose to debug BF failure.

  7. snapshot scalability: Introduce dense array of in-progress xids.

  8. snapshot scalability: Move PGXACT->vacuumFlags to ProcGlobal->vacuumFlags.

  9. snapshot scalability: Move subxact info to ProcGlobal, remove PGXACT.

  10. snapshot scalability: Move PGXACT->xmin back to PGPROC.

  11. snapshot scalability: Don't compute global horizons while building snapshots.

  12. BRIN: Handle concurrent desummarization properly

  13. Track latest completed xid as a FullTransactionId.

  14. Rename VariableCacheData.nextFullXid to nextXid.

  15. snapshot scalability: Move delayChkpt from PGXACT to PGPROC.

  16. Report progress of CREATE INDEX operations