Horton

Re: Handing off SLRU fsyncs to the checkpointer

Jakub Wartak <jakub.wartak@tomtom.com>

View thread View on postgr.es
From: Jakub Wartak <Jakub.Wartak@tomtom.com>
To: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Thomas Munro <thomas.munro@gmail.com>, Robert Haas <robertmhaas@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2020-08-27T10:13:46Z
Lists: pgsql-hackers 
Hi Alvaro, Thomas, hackers

>>     14.69%  postgres  postgres            [.] hash_search_with_hash_value
>>             ---hash_search_with_hash_value
>>                |--9.80%--BufTableLookup
[..]
>>                 --4.90%--smgropen
>>                           |--2.86%--ReadBufferWithoutRelcache
> Looking at an earlier report of this problem I was thinking whether it'd
> make sense to replace SMgrRelationHash with a simplehash table; I have a
> half-written patch for that, but I haven't completed that work.
> However, in the older profile things were looking different, as
> hash_search_with_hash_value was taking 35.25%, and smgropen was 33.74%
> of it.  BufTableLookup was also there but only 1.51%.  So I'm not so
> sure now that that'll pay off as clearly as I had hoped.

Yes, quite frankly my expectation was to see hash_search_with_hash_value()<-smgropen() outcome as 1st one, but in simplified redo-bench script it's not the case. The original scenario was much more complex with plenty of differences (in no particular order: TB-sized DB VS ~500GB RAM -> thousands of forks, multiple tables, huge btrees, multiple INSERTs wih plenty of data in VALUES() thrown as one commit, real primary->hot-standby replication [not closed DB in recovery], sorted not random UUIDs) - I'm going to try nail down these differences and maybe I manage to produce more realistic "pgbench reproducer" (this may take some time though).

-Jakub Wartak.

Commits

  1. Remove unused function prototypes.

  2. Defer flushing of SLRU files.

  3. Improve the vacuum error context phase information.

  4. Cache smgrnblocks() results in recovery.

  5. Refactor the fsync queue for wider use.

  6. Increase maximum number of clog buffers.

  7. Make the number of CLOG buffers adaptive, based on shared_buffers.

  8. Replace implementation of pg_log as a relation accessed through the