Re: Handing off SLRU fsyncs to the checkpointer

Alvaro Herrera <alvherre@2ndquadrant.com>

From: Alvaro Herrera <alvherre@2ndquadrant.com>
To: Jakub Wartak <Jakub.Wartak@tomtom.com>
Cc: Thomas Munro <thomas.munro@gmail.com>, Robert Haas <robertmhaas@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2020-08-26T18:15:32Z
Lists: pgsql-hackers
On 2020-Aug-25, Jakub Wartak wrote:

> Turning on/off the defer SLRU patch and/or fsync doesn't seem to make
> any difference, so if anyone is curious the next sets of append-only
> bottlenecks is like below:
> 
>     14.69%  postgres  postgres            [.] hash_search_with_hash_value
>             ---hash_search_with_hash_value
>                |--9.80%--BufTableLookup
>                |          ReadBuffer_common
>                |          ReadBufferWithoutRelcache
>                |          XLogReadBufferExtended
>                |          XLogReadBufferForRedoExtended
>                |          |--7.76%--btree_xlog_insert
>                |          |          btree_redo
>                |          |          StartupXLOG
>                |           --1.63%--heap_xlog_insert
>                 --4.90%--smgropen
>                           |--2.86%--ReadBufferWithoutRelcache

Looking at an earlier report of this problem I was thinking whether it'd
make sense to replace SMgrRelationHash with a simplehash table; I have a
half-written patch for that, but I haven't completed that work.
However, in the older profile things were looking different, as
hash_search_with_hash_value was taking 35.25%, and smgropen was 33.74%
of it.  BufTableLookup was also there but only 1.51%.  So I'm not so
sure now that that'll pay off as clearly as I had hoped.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Commits

  1. Remove unused function prototypes.

  2. Defer flushing of SLRU files.

  3. Improve the vacuum error context phase information.

  4. Cache smgrnblocks() results in recovery.

  5. Refactor the fsync queue for wider use.

  6. Increase maximum number of clog buffers.

  7. Make the number of CLOG buffers adaptive, based on shared_buffers.

  8. Replace implementation of pg_log as a relation accessed through the