Horton

Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

Magnus Hagander <magnus@hagander.net>

View thread View on postgr.es
From: Magnus Hagander <magnus@hagander.net>
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: Fujii Masao <masao.fujii@oss.nttdata.com>, "Shinoda, Noriyoshi (PN Japan A&PS Delivery)" <noriyoshi.shinoda@hpe.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, "amitlangote09@gmail.com" <amitlangote09@gmail.com>, "masahiko.sawada@2ndquadrant.com" <masahiko.sawada@2ndquadrant.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2020-03-05T15:45:24Z
Lists: pgsql-hackers 
On Wed, Mar 4, 2020 at 11:15 PM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
>
> On 2020-03-05 05:53, Fujii Masao wrote:
> > Or, as another approach, it might be worth considering to make
> > the server always estimate the total backup size whether --progress is
> > specified or not, as Amit argued upthread. If the time required to
> > estimate the backup size is negligible compared to total backup time,
> > IMO this approach seems better. If we adopt this, we can also get
> > rid of PROGESS option from BASE_BACKUP replication command.
>
> I think that would be preferable.

From a UI perspective I definitely agree.

The problem with that one is that it can take a non-trivlal amount of
time, that's why it was made an option (in the protocol) in the first
place. Particularly if you have a database with many small objets.

Is it enough to care about? I'm not sure, but it's definitely
something to consider. It was not negligible in some tests I ran then,
but it is quite some time ago and reality has definitely changed since
then.

-- 
 Magnus Hagander
 Me: https://www.hagander.net/
 Work: https://www.redpill-linpro.com/

Commits

  1. Report NULL as total backup size if it's not estimated.

  2. Make pg_basebackup ask the server to estimate the total backup size, by default.

  3. Report progress of streaming base backup.