Horton

Re: Duplicate history file?

Kyotaro Horiguchi <horikyota.ntt@gmail.com>

View thread View on postgr.es
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: tatsuro.yamada.tf@nttcom.co.jp
Cc: pgsql-hackers@lists.postgresql.org
Date: 2021-06-07T07:38:01Z
Lists: pgsql-hackers 
At Mon, 07 Jun 2021 15:57:00 +0900, Tatsuro Yamada <tatsuro.yamada.tf@nttcom.co.jp> wrote in 
> Hi Horiguchi-san,
> 
> >> Regarding "test ! -f",
> >> I am wondering how many people are using the test command for
> >> archive_command. If I remember correctly, the guide provided by
> >> NTT OSS Center that we are using does not recommend using the test
> >> command.
> > I think, as the PG-REX documentation says, the simple cp works well as
> > far as the assumption of PG-REX - no double failure happenes, and
> > following the instruction - holds.
> 
> 
> I believe that this assumption started to be wrong after
> archive_mode=always was introduced. As far as I can tell, it doesn't
> happen when it's archive_mode=on.

?? Doesn't *simple* cp (without "test") work for you?  I meant that
the operating assumption of PG-REX ensures that overwriting doesn't
cause a problem.

> > Otherwise the documentation would need someting like the following if
> > we assume the current behavior.
> > 
> >> The archive command should generally be designed to refuse to
> >> overwrite any pre-existing archive file. This is an important safety
> >> feature to preserve the integrity of your archive in case of
> >> administrator error (such as sending the output of two different
> >> servers to the same archive directory).
> > + For standby with the setting archive_mode=always, there's a case
> > where
> > + the same file is archived more than once.  For safety, it is
> > + recommended that when the destination file exists, the
> > archive_command
> > + returns zero if it is byte-identical to the source file.
> 
> 
> Agreed.
> That is same solution as I mentioned earlier.
> If possible, it also would better to write it postgresql.conf (that
> might
> be overkill?!).

Mmmm, I didn't noticed that. I don't think such a complex caveat fits
the configuration file. And if we need such a caveart there, it might
be the sign that we need to fix the causal behavior...

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

Commits

  1. Further refinement of stuck_on_old_timeline recovery test

  2. Adjust new test case to set wal_keep_segments.

  3. Adjust new test case to set wal_keep_size.

  4. Fix corner case failure of new standby to follow new primary.

  5. Back-port a few PostgresNode.pm methods.

  6. Allow PostgresNode.pm's backup method to accept backup_options.

  7. Add end-to-end testing of pg_basebackup's tar-format output.

  8. Allow PostgresNode.pm tests to wait for catchup

  9. Delay reading timeline history file until it's fetched from master.