Re: remove more archiving overhead

Nathan Bossart <nathandbossart@gmail.com>

From: Nathan Bossart <nathandbossart@gmail.com>
To: David Steele <david@pgmasters.net>
Cc: Robert Haas <robertmhaas@gmail.com>, Fujii Masao <masao.fujii@oss.nttdata.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2022-07-07T17:51:42Z
Lists: pgsql-hackers

Attachments

On Thu, Jul 07, 2022 at 09:18:25AM -0700, Nathan Bossart wrote:
> On Thu, Jul 07, 2022 at 10:46:23AM -0400, David Steele wrote:
>> On 7/7/22 10:37, Robert Haas wrote:
>>> I don't object, but I just started to wonder whether the need to
>>> handle re-archiving of the same file cleanly is as well-documented as
>>> it ought to be.
>> 
>> +1, but I don't think that needs to stand in the way of this patch, which
>> looks sensible to me as-is. I think that's what you meant, but just wanted
>> to be sure.
> 
> Yeah, this seems like something that should be documented.  I can pick this
> up.  I believe this is an existing problem, but this patch could make it
> more likely.

Here is a first try at documenting this.  I'm not thrilled about the
placement, since it feels a bit buried in the backup docs, but this is
where this sort of thing lives today.  It also seems odd to stress the
importance of avoiding overwriting pre-existing archives in case multiple
servers are archiving to the same place while only offering solutions with
obvious race conditions.  Even basic_archive is subject to this now that
durable_rename_excl() no longer exists.  Perhaps we should make a note of
that, too.

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

Commits

  1. Be more prescriptive in new doc about re-archiving of same WAL file.

  2. doc: Add note about re-archiving of same WAL files in docs.

  3. Reduce overhead of renaming archive status files.