Horton

Re: Weird failure with latches in curculio on v15

Nathan Bossart <nathandbossart@gmail.com>

View thread View on postgr.es
From: Nathan Bossart <nathandbossart@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Michael Paquier <michael@paquier.xyz>, Tom Lane <tgl@sss.pgh.pa.us>, Thomas Munro <thomas.munro@gmail.com>, Fujii Masao <fujii@postgresql.org>, Postgres hackers <pgsql-hackers@lists.postgresql.org>
Date: 2023-02-06T00:46:31Z
Lists: pgsql-hackers 
On Sun, Feb 05, 2023 at 04:07:50PM -0800, Andres Freund wrote:
> On 2023-02-05 15:57:47 -0800, Nathan Bossart wrote:
>> I agree that the shell overhead isn't the main performance issue,
>> but it's unclear to me how much of this should be baked into
>> PostgreSQL.
> 
> I don't know fully either. But just reimplementing all of it in
> different modules doesn't seem like a sane approach either. A lot of it
> is policy that we need to solve once, centrally.
> 
>> I mean, we could introduce a GUC that tells us how far ahead to
>> restore and have a background worker (or multiple background workers)
>> asynchronously pull files into a staging directory via the callbacks.
>> Is that the sort of scope you are envisioning?
> 
> Closer, at least.

Got it.  I suspect we'll want to do something similar for archive modules
eventually, too.

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

Commits

  1. Avoid calling proc_exit() in processes forked by system().

  2. Move extra code out of the Pre/PostRestoreCommand() section.

  3. Revert refactoring of restore command code to shell_restore.c

  4. Refactor code in charge of running shell-based recovery commands

  5. Clean up inconsistent use of fflush().

  6. Report wait events for local shell commands like archive_command.