Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

Fujii Masao <masao.fujii@oss.nttdata.com>

From: Fujii Masao <masao.fujii@oss.nttdata.com>
To: Magnus Hagander <magnus@hagander.net>
Cc: Amit Langote <amitlangote09@gmail.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>, "Shinoda, Noriyoshi (PN Japan A&PS Delivery)" <noriyoshi.shinoda@hpe.com>, Masahiko Sawada <masahiko.sawada@2ndquadrant.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2020-03-18T16:13:59Z
Lists: pgsql-hackers

Attachments


On 2020/03/19 0:37, Magnus Hagander wrote:
> On Wed, Mar 11, 2020 at 5:53 AM Fujii Masao <masao.fujii@oss.nttdata.com> wrote:
>>
>>
>>
>> On 2020/03/11 3:39, Magnus Hagander wrote:
>>> On Tue, Mar 10, 2020 at 6:19 PM Fujii Masao <masao.fujii@oss.nttdata.com> wrote:
>>>>
>>>>
>>>>
>>>> On 2020/03/10 22:43, Amit Langote wrote:
>>>>> On Tue, Mar 10, 2020 at 6:09 PM Fujii Masao <masao.fujii@oss.nttdata.com> wrote:
>>>>>>> So, I will make the patch adding support for --no-estimate-size option
>>>>>>> in pg_basebackup.
>>>>>>
>>>>>> Patch attached.
>>>>>
>>>>> Like the idea and the patch looks mostly good.
>>>>
>>>> Thanks for reviewing the patch!
>>>>
>>>>> +      total size. If the estimation is disabled in
>>>>> +      <application>pg_basebackup</application>
>>>>> +      (i.e., <literal>--no-estimate-size</literal> option is specified),
>>>>> +      this is always <literal>0</literal>.
>>>>>
>>>>> "always" seems unnecessary.
>>>>
>>>> Fixed.
>>>>
>>>>> +        This option prevents the server from estimating the total
>>>>> +        amount of backup data that will be streamed. In other words,
>>>>> +        <literal>backup_total</literal> column in the
>>>>> +        <structname>pg_stat_progress_basebackup</structname>
>>>>> +        view always indicates <literal>0</literal> if this option is enabled.
>>>>>
>>>>> Here too.
>>>>
>>>> Fixed.
>>>>
>>>> Attached is the updated version of the patch.
>>>
>>> Would it perhaps be better to return NULL instead of 0 in the
>>> statistics view if there is no data?
> 
> Did you miss this comment, or not agree? :)

Oh, I forgot to attached the patch... Patch attached.
This patch needs to be applied after applying
add_no_estimate_size_v3.patch.

>>> Also, should it really  be the server version that decides how this
>>> feature behaves, and not the pg_basebackup version? Given that the
>>> implementation is entirely in the client, it seems that's more
>>> logical?
>>
>> Yeah, you're right. I changed the patch that way.
>> Attached is the updated version of the patch.
> 
> The other changes in it look good!

Thanks for the review!

Regards,

-- 
Fujii Masao
NTT DATA CORPORATION
Advanced Platform Technology Group
Research and Development Headquarters

Commits

  1. Report NULL as total backup size if it's not estimated.

  2. Make pg_basebackup ask the server to estimate the total backup size, by default.

  3. Report progress of streaming base backup.