RE: Background writer and checkpointer in crash recovery

Jakub Wartak <jakub.wartak@tomtom.com>

From: Jakub Wartak <Jakub.Wartak@tomtom.com>
To: Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>
Cc: Aleksander Alekseev <aleksander@timescale.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Thomas Munro <thomas.munro@gmail.com>
Date: 2021-08-02T13:51:44Z
Lists: pgsql-hackers
> On Fri, Jul 30, 2021 at 4:00 PM Andres Freund <andres@anarazel.de> wrote:
> > I don't agree with that? If (user+system) << wall then it is very
> > likely that recovery is IO bound. If system is a large percentage of
> > wall, then shared buffers is likely too small (or we're replacing the
> > wrong
> > buffers) because you spend a lot of time copying data in/out of the
> > kernel page cache. If user is the majority, you're CPU bound.
> >
> > Without user & system time it's much harder to figure that out - at
> > least for me.
> 
> Oh, that's an interesting point. At least now I'll know why I am supposed to care
> about that log line the next time I see it. I guess we could include both things,
> though the line might get a little long.
> Or maybe there's some other subset that would make sense.

Hi Robert,

The email threads from [1] can serve as indication that having complete view of 
resource usage (user+system+elapsed) is advantageous in different situations and 
pretty platform-generic. Also as  Andres and Simon earlier pointed out - the performance
insight into crash recovery/replication performance is next to nothing, judging just by 
looking at currently emitted log messages. The more the there is, the better I think. 

BTW,  if you now there's this big push for refactoring StartupXLOG() then what 
frustrating^H^H^H^H^H could be done better - at least from end-user point of view -  
is that there is lack of near real time cyclic messages (every 1min?) about current status, 
performance and maybe even ETA (simplistic case; assuming it is linear). 

[1] - https://commitfest.postgresql.org/30/2799/ 



Commits

  1. Clear ps display of startup process at the end of recovery

  2. Further simplify a bit of logic in StartupXLOG().

  3. Run checkpointer and bgwriter in crash recovery.

  4. Add some checkpoint/restartpoint status to ps display

  5. Start background writer during archive recovery. Background writer now performs