Horton

Re: Weird failure with latches in curculio on v15

Robert Haas <robertmhaas@gmail.com>

View thread View on postgr.es
From: Robert Haas <robertmhaas@gmail.com>
To: Nathan Bossart <nathandbossart@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>, Michael Paquier <michael@paquier.xyz>, Thomas Munro <thomas.munro@gmail.com>, Fujii Masao <fujii@postgresql.org>, Postgres hackers <pgsql-hackers@lists.postgresql.org>
Date: 2023-03-01T17:44:51Z
Lists: pgsql-hackers 
On Tue, Feb 28, 2023 at 11:29 PM Nathan Bossart
<nathandbossart@gmail.com> wrote:
> On Sat, Feb 25, 2023 at 11:00:31AM -0800, Andres Freund wrote:
> > TBH, I think the current archive and restore module APIs aren't useful. I
> > think it was a mistake to add archive modules without having demonstrated that
> > one can do something useful with them that the restore_command didn't already
> > do. If anything, archive modules have made it harder to improve archiving
> > performance via concurrency.
>
> I must respectfully disagree that this work is useless.  Besides the
> performance and security benefits of not shelling out for every WAL file,
> I've found it very useful to be able to use the standard module framework
> to develop archive modules.  It's relatively easy to make use of GUCs,
> background workers, compression, etc.  Of course, there is room for
> improvement in areas like concurrency support as you rightly point out, but
> I don't think that makes the current state worthless.

I also disagree with Andres. The status quo ante was that we did not
provide any way of doing archiving correctly even to a directory on
the local machine. We could only recommend silly things like 'cp' that
are incorrect in multiple ways. basic_archive isn't the most wonderful
thing ever, and its deficiencies are more obvious to me now than they
were when I committed it. But it's better than recommending a shell
command that doesn't even work.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

Commits

  1. Avoid calling proc_exit() in processes forked by system().

  2. Move extra code out of the Pre/PostRestoreCommand() section.

  3. Revert refactoring of restore command code to shell_restore.c

  4. Refactor code in charge of running shell-based recovery commands

  5. Clean up inconsistent use of fflush().

  6. Report wait events for local shell commands like archive_command.