Horton

Re: Improving connection scalability: GetSnapshotData()

Konstantin Knizhnik <k.knizhnik@postgrespro.ru>

View thread View on postgr.es
From: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
To: Andres Freund <andres@anarazel.de>
Cc: pgsql-hackers@lists.postgresql.org
Date: 2020-09-07T14:20:53Z
Lists: pgsql-hackers

Attachments


On 06.09.2020 21:56, Andres Freund wrote:
>
> Hm, that is interesting / odd. If you record a profile with call graphs
> (e.g. --call-graph dwarf), where are all the LWLockAttemptLock calls
> comming from?
>
Attached.

> I assume the machine you're talking about is an 8 socket machine?
>
> What if you:
> a) start postgres and pgbench with numactl --interleave=all
> b) start postgres with numactl --interleave=0,1 --cpunodebind=0,1 --membind=0,1
>     in case you have 4 sockets, or 0,1,2,3 in case you have 8 sockets?
>

TPS for -c 100

--interleave=all
1168910
--interleave=0,1
1232557
--interleave=0,1,2,3
1254271
--cpunodebind=0,1,2,3 --membind=0,1,2,3
1237237
--cpunodebind=0,1 --membind=0,1
1420211
--cpunodebind=0 --membind=0
1101203


>> And which pgbench database scale factor you have used?
> 200
>
> Another thing you could try is to run 2-4 pgench instances in different
> databases.
I tried to reinitialize database with scale 200 but there was no 
significant improvement in performance.

Commits

  1. Try to unbreak 021_row_visibility.pl on mingw.

  2. Fix and test snapshot behavior on standby.

  3. Fix race condition in snapshot caching when 2PC is used.

  4. snapshot scalability: cache snapshots using a xact completion counter.

  5. Fix use of wrong index in ComputeXidHorizons().

  6. Make vacuum a bit more verbose to debug BF failure.

  7. snapshot scalability: Introduce dense array of in-progress xids.

  8. snapshot scalability: Move PGXACT->vacuumFlags to ProcGlobal->vacuumFlags.

  9. snapshot scalability: Move subxact info to ProcGlobal, remove PGXACT.

  10. snapshot scalability: Move PGXACT->xmin back to PGPROC.

  11. snapshot scalability: Don't compute global horizons while building snapshots.

  12. BRIN: Handle concurrent desummarization properly

  13. Track latest completed xid as a FullTransactionId.

  14. Rename VariableCacheData.nextFullXid to nextXid.

  15. snapshot scalability: Move delayChkpt from PGXACT to PGPROC.

  16. Report progress of CREATE INDEX operations