Horton

Thread

Commits

  1. doc: Fix obsolete description about pg_basebackup.

  2. Make pg_stop_backup's wait_for_archive flag work on standbys.

  1. Stale description for pg_basebackup

    Kyotaro Horiguchi <horikyota.ntt@gmail.com> — 2021-04-20T04:32:35Z

    Hello.

It seems to me that there's a stale description in the documentation
of pg_basebackup.

https://www.postgresql.org/docs/13/app-pgbasebackup.html

> Note that there are some limitations in taking a backup from a standby:
...
> If you are using -X none, there is no guarantee that all WAL files
> required for the backup are archived at the end of backup.

Actually, pg_basebackup waits for the all required files to be
archived, which is an established behavior by commit
52f8a59dd9@PG10. However, the same commit seems to have forgot to
change the doc for pg_basebackup. (The current description is
introduced by 9a4d51077c@PG10)

The attached is a proposal to rewrite it as the following.

+ If you are using -X none, pg_basebackup may wait for a long time for
+ all the required WAL files to be archived. In that case, You may need
+ to call pg_switch_wal() on the primary to complete it sooner.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

  2. Re: Stale description for pg_basebackup

    Kyotaro Horiguchi <horikyota.ntt@gmail.com> — 2021-04-21T01:43:30Z

    At Tue, 20 Apr 2021 13:32:35 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in 
> Hello.
> 
> It seems to me that there's a stale description in the documentation
> of pg_basebackup.
> 
> https://www.postgresql.org/docs/13/app-pgbasebackup.html
> 
> > Note that there are some limitations in taking a backup from a standby:
> ...
> > If you are using -X none, there is no guarantee that all WAL files
> > required for the backup are archived at the end of backup.
> 
> Actually, pg_basebackup waits for the all required files to be
> archived, which is an established behavior by commit
> 52f8a59dd9@PG10. However, the same commit seems to have forgot to
> change the doc for pg_basebackup. (The current description is
> introduced by 9a4d51077c@PG10)
> 
> The attached is a proposal to rewrite it as the following.
> 
> + If you are using -X none, pg_basebackup may wait for a long time for
> + all the required WAL files to be archived. In that case, You may need
> + to call pg_switch_wal() on the primary to complete it sooner.

I forgot to preserve the description about *primary*. It should be as
the following instead.

+ If you are using -X none, there is no guarantee on the primary that
+ all WAL files required for the backup are archived at the end of
+ backup. When the standby is configured as archive_mode=always,
+ pg_basebackup may wait for a long time for all the required WAL files
+ to be archived. In that case, You may need to call pg_switch_wal() on
+ the primary to complete it sooner.

Attached.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

  3. Re: Stale description for pg_basebackup

    Kyotaro Horiguchi <horikyota.ntt@gmail.com> — 2021-04-21T02:09:37Z

    At Wed, 21 Apr 2021 10:43:30 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in 
> At Tue, 20 Apr 2021 13:32:35 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in 
> > Hello.
> > 
> > It seems to me that there's a stale description in the documentation
> > of pg_basebackup.
> > 
> > https://www.postgresql.org/docs/13/app-pgbasebackup.html
> > 
> > > Note that there are some limitations in taking a backup from a standby:
> > ...
> > > If you are using -X none, there is no guarantee that all WAL files
> > > required for the backup are archived at the end of backup.
> > 
> > Actually, pg_basebackup waits for the all required files to be
> > archived, which is an established behavior by commit
> > 52f8a59dd9@PG10. However, the same commit seems to have forgot to
> > change the doc for pg_basebackup. (The current description is
> > introduced by 9a4d51077c@PG10)
> > 
> > The attached is a proposal to rewrite it as the following.
> > 
> > + If you are using -X none, pg_basebackup may wait for a long time for
> > + all the required WAL files to be archived. In that case, You may need
> > + to call pg_switch_wal() on the primary to complete it sooner.
> 
> I forgot to preserve the description about *primary*. It should be as
> the following instead.
> 
> + If you are using -X none, there is no guarantee on the primary that
> + all WAL files required for the backup are archived at the end of
> + backup. When the standby is configured as archive_mode=always,
> + pg_basebackup may wait for a long time for all the required WAL files
> + to be archived. In that case, You may need to call pg_switch_wal() on
> + the primary to complete it sooner.

Hmm. Some words need to be qualified. Attached.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

  4. Re: Stale description for pg_basebackup

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2021-04-21T14:06:56Z

    
On 2021/04/21 11:09, Kyotaro Horiguchi wrote:
> At Wed, 21 Apr 2021 10:43:30 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
>> At Tue, 20 Apr 2021 13:32:35 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
>>> Hello.
>>>
>>> It seems to me that there's a stale description in the documentation
>>> of pg_basebackup.

I think you're right.


> Hmm. Some words need to be qualified. Attached.

+       If you are using <literal>-X none</literal>, there is no guarantee on
+       the primary that all WAL files required for the backup are archived at
+       the end of backup.

I don't think that this should be picked up as a limitation of standby backup.
Because users basically want to make pg_basebackup wait for all required
WAL files to be archived on the standby, in the standby backup case.


When <varname>archive_mode</varname> is set
+       to <literal>on</literal> on the

"on" should be "always"?


+       standby, <application>pg_basebackup</application> may wait for a long
+       time for all the required WAL files to be archived.  In that case, You
+       may need to call <function>pg_switch_wal()</function> on the primary to
+       complete it sooner.

What about the following description?

-------------------
When you are using -X none, if write activity on the primary is low,
pg_basebackup may need to wait a long time for all WAL files required for
the backup to be archived. It may be useful to run pg_switch_wal
on the primary in order to trigger an immediate WAL file switch and archiving.
-------------------


Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

  5. Re: Stale description for pg_basebackup

    Kyotaro Horiguchi <horikyota.ntt@gmail.com> — 2021-04-22T00:25:33Z

    Ugg. I was confused.


At Wed, 21 Apr 2021 23:06:56 +0900, Fujii Masao <masao.fujii@oss.nttdata.com> wrote in 
> > Hmm. Some words need to be qualified. Attached.
> 
> + If you are using <literal>-X none</literal>, there is no guarantee
> on
> + the primary that all WAL files required for the backup are archived
> at
> +       the end of backup.
> 
> I don't think that this should be picked up as a limitation of standby
> backup.
> Because users basically want to make pg_basebackup wait for all
> required
> WAL files to be archived on the standby, in the standby backup case.

Yeah, you're right. I think it is what I thought at first. The last
proposal is a result of some confusion..

> When <varname>archive_mode</varname> is set
> +       to <literal>on</literal> on the
> 
> "on" should be "always"?

Yes..

> + standby, <application>pg_basebackup</application> may wait for a
> long
> + time for all the required WAL files to be archived.  In that case,
> You
> + may need to call <function>pg_switch_wal()</function> on the primary
> to
> +       complete it sooner.
> 
> What about the following description?
> 
> -------------------
> When you are using -X none, if write activity on the primary is low,
> pg_basebackup may need to wait a long time for all WAL files required
> for
> the backup to be archived. It may be useful to run pg_switch_wal
> on the primary in order to trigger an immediate WAL file switch and
> archiving.
> -------------------

Looks far better.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

  6. Re: Stale description for pg_basebackup

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2021-04-22T01:56:10Z

    
On 2021/04/22 9:25, Kyotaro Horiguchi wrote:
>> What about the following description?
>>
>> -------------------
>> When you are using -X none, if write activity on the primary is low,
>> pg_basebackup may need to wait a long time for all WAL files required
>> for
>> the backup to be archived. It may be useful to run pg_switch_wal
>> on the primary in order to trigger an immediate WAL file switch and
>> archiving.
>> -------------------
> 
> Looks far better.

Patch attached. I appended the following description to assist
users to understand why pg_basebackup may need wait a long time
when write activity is low on the primary.

------------------
pg_basebackup cannot force the standby to switch to
a new WAL file at the end of backup.
------------------

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

  7. Re: Stale description for pg_basebackup

    Kyotaro Horiguchi <horikyota.ntt@gmail.com> — 2021-04-22T02:19:40Z

    At Thu, 22 Apr 2021 10:56:10 +0900, Fujii Masao <masao.fujii@oss.nttdata.com> wrote in 
> 
> 
> On 2021/04/22 9:25, Kyotaro Horiguchi wrote:
> >> What about the following description?
> >>
> >> -------------------
> >> When you are using -X none, if write activity on the primary is low,
> >> pg_basebackup may need to wait a long time for all WAL files required
> >> for
> >> the backup to be archived. It may be useful to run pg_switch_wal
> >> on the primary in order to trigger an immediate WAL file switch and
> >> archiving.
> >> -------------------
> > Looks far better.
> 
> Patch attached. I appended the following description to assist
> users to understand why pg_basebackup may need wait a long time
> when write activity is low on the primary.
> 
> ------------------
> pg_basebackup cannot force the standby to switch to
> a new WAL file at the end of backup.
> ------------------

I'm not sure which is the convention here, but I saw that some
function names in the doc are followed by parentheses (ie
pg_switch_wal()).

(prepended?) It seems a bit redundant but also a bit clearer. How
about the following simplification?

- It may be useful to run pg_switch_wal on the primary in order to
- trigger an immediate WAL file switch and archiving.
+ It may be useful to run pg_switch_wal() on the primary in that case.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

  8. Re: Stale description for pg_basebackup

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2021-04-22T04:06:50Z

    
On 2021/04/22 11:19, Kyotaro Horiguchi wrote:
> At Thu, 22 Apr 2021 10:56:10 +0900, Fujii Masao <masao.fujii@oss.nttdata.com> wrote in
>>
>>
>> On 2021/04/22 9:25, Kyotaro Horiguchi wrote:
>>>> What about the following description?
>>>>
>>>> -------------------
>>>> When you are using -X none, if write activity on the primary is low,
>>>> pg_basebackup may need to wait a long time for all WAL files required
>>>> for
>>>> the backup to be archived. It may be useful to run pg_switch_wal
>>>> on the primary in order to trigger an immediate WAL file switch and
>>>> archiving.
>>>> -------------------
>>> Looks far better.
>>
>> Patch attached. I appended the following description to assist
>> users to understand why pg_basebackup may need wait a long time
>> when write activity is low on the primary.
>>
>> ------------------
>> pg_basebackup cannot force the standby to switch to
>> a new WAL file at the end of backup.
>> ------------------
> 
> I'm not sure which is the convention here, but I saw that some
> function names in the doc are followed by parentheses (ie
> pg_switch_wal()).

Either works for me. I didn't add "()" because I just used the same description
as that in func.sgml.

         it may be useful to run <function>pg_switch_wal</function> on the
         primary in order to trigger an immediate segment switch.)


> (prepended?) It seems a bit redundant but also a bit clearer. How
> about the following simplification?
> 
> - It may be useful to run pg_switch_wal on the primary in order to
> - trigger an immediate WAL file switch and archiving.
> + It may be useful to run pg_switch_wal() on the primary in that case.

IMO "in order to..." part is helpful for us to understand why pg_switch_wal
is useful in this case. So I'd like to leave it.

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

  9. Re: Stale description for pg_basebackup

    Kyotaro Horiguchi <horikyota.ntt@gmail.com> — 2021-04-22T04:25:27Z

    At Thu, 22 Apr 2021 13:06:50 +0900, Fujii Masao <masao.fujii@oss.nttdata.com> wrote in 
> Either works for me. I didn't add "()" because I just used the same
> description
> as that in func.sgml.
> 
>         it may be useful to run <function>pg_switch_wal</function> on the
>         primary in order to trigger an immediate segment switch.)
..
> IMO "in order to..." part is helpful for us to understand why
> pg_switch_wal
> is useful in this case. So I'd like to leave it.

Ok, I'm fine with both of them. Thanks for the explanation.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

  10. Re: Stale description for pg_basebackup

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2021-04-23T06:53:02Z

    
On 2021/04/22 13:25, Kyotaro Horiguchi wrote:
> At Thu, 22 Apr 2021 13:06:50 +0900, Fujii Masao <masao.fujii@oss.nttdata.com> wrote in
>> Either works for me. I didn't add "()" because I just used the same
>> description
>> as that in func.sgml.
>>
>>          it may be useful to run <function>pg_switch_wal</function> on the
>>          primary in order to trigger an immediate segment switch.)
> ..
>> IMO "in order to..." part is helpful for us to understand why
>> pg_switch_wal
>> is useful in this case. So I'd like to leave it.
> 
> Ok, I'm fine with both of them. Thanks for the explanation.

Pushed. Thanks!

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION