Horton

Re: Improving connection scalability: GetSnapshotData()

Konstantin Knizhnik <k.knizhnik@postgrespro.ru>

View thread View on postgr.es
From: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
To: Andres Freund <andres@anarazel.de>
Cc: pgsql-hackers@lists.postgresql.org
Date: 2020-09-06T11:05:35Z
Lists: pgsql-hackers 

On 04.09.2020 21:53, Andres Freund wrote:
>
>> May be it is because of more complex architecture of my server?
> Think we'll need profiles to know...

This is "perf top" of pgebch -c 100 -j 100 -M prepared -S

   12.16%  postgres                           [.] PinBuffer
   11.92%  postgres                           [.] LWLockAttemptLock
    6.46%  postgres                           [.] UnpinBuffer.constprop.11
    6.03%  postgres                           [.] LWLockRelease
    3.14%  postgres                           [.] BufferGetBlockNumber
    3.04%  postgres                           [.] ReadBuffer_common
    2.13%  [kernel]                           [k] _raw_spin_lock_irqsave
    2.11%  [kernel]                           [k] switch_mm_irqs_off
    1.95%  postgres                           [.] _bt_compare


Looks like most of the time is pent in buffers locks.
And which pgbench database scale factor you have used?

Commits

  1. Try to unbreak 021_row_visibility.pl on mingw.

  2. Fix and test snapshot behavior on standby.

  3. Fix race condition in snapshot caching when 2PC is used.

  4. snapshot scalability: cache snapshots using a xact completion counter.

  5. Fix use of wrong index in ComputeXidHorizons().

  6. Make vacuum a bit more verbose to debug BF failure.

  7. snapshot scalability: Introduce dense array of in-progress xids.

  8. snapshot scalability: Move PGXACT->vacuumFlags to ProcGlobal->vacuumFlags.

  9. snapshot scalability: Move subxact info to ProcGlobal, remove PGXACT.

  10. snapshot scalability: Move PGXACT->xmin back to PGPROC.

  11. snapshot scalability: Don't compute global horizons while building snapshots.

  12. BRIN: Handle concurrent desummarization properly

  13. Track latest completed xid as a FullTransactionId.

  14. Rename VariableCacheData.nextFullXid to nextXid.

  15. snapshot scalability: Move delayChkpt from PGXACT to PGPROC.

  16. Report progress of CREATE INDEX operations