Horton

Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

Fujii Masao <masao.fujii@oss.nttdata.com>

View thread View on postgr.es
From: Fujii Masao <masao.fujii@oss.nttdata.com>
To: Andres Freund <andres@anarazel.de>
Cc: Magnus Hagander <magnus@hagander.net>, Amit Langote <amitlangote09@gmail.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>, "Shinoda, Noriyoshi (PN Japan A&PS Delivery)" <noriyoshi.shinoda@hpe.com>, Masahiko Sawada <masahiko.sawada@2ndquadrant.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2020-03-23T07:06:46Z
Lists: pgsql-hackers 

On 2020/03/20 3:39, Andres Freund wrote:
> Hi,
> 
> On 2020-03-19 17:21:38 +0900, Fujii Masao wrote:
>> Pushed! Thanks!
> 
> FWIW, I'm a bit doubtful that incuring the overhead of this by default
> on everybody is a nice thing. On filesystems with high latency and with
> a lot of small relations the overhead of stating a lot of files can be
> almost as high as the actual base backup.

Yeah, so if we receive lots of complaints like that during beta and
RC phases, we should consider to change the default behavior.

Also maybe I should measure how long the estimation takes on the env
where, for example, ten thousand tables (i.e., files) exist, in order to
whether the default behavior is really time-consuming or not?

Regards,

-- 
Fujii Masao
NTT DATA CORPORATION
Advanced Platform Technology Group
Research and Development Headquarters

Commits

  1. Report NULL as total backup size if it's not estimated.

  2. Make pg_basebackup ask the server to estimate the total backup size, by default.

  3. Report progress of streaming base backup.