Re: [PoC] pg_upgrade: allow to upgrade publisher node

Amit Kapila <amit.kapila16@gmail.com>

From: Amit Kapila <amit.kapila16@gmail.com>
To: Julien Rouhaud <rjuju123@gmail.com>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, vignesh C <vignesh21@gmail.com>
Date: 2023-08-11T05:48:09Z
Lists: pgsql-hackers
On Fri, Aug 11, 2023 at 10:43 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
>
> On Thu, Aug 10, 2023 at 04:30:40PM +0900, Masahiko Sawada wrote:
> > On Thu, Aug 10, 2023 at 2:27 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > >
> > > Sawada-San, Julien, and others, do you have any thoughts on the above point?
> >
> > IIUC during the old cluster running in the middle of pg_upgrade it
> > doesn't accept TCP connections. I'm not sure we need to worry about
> > the case where someone in the same server attempts to create
> > replication slots during the upgrade.
>
> AFAICS this is only true for non-Windows platform, so we would still need some
> extra safeguards on Windows.  Having those on all platforms will probably be
> simpler and won't hurt otherwise.
>
> > The same is true for other objects, as Amit mentioned.
>
> I disagree.  As I mentioned before any module registered in
> shared_preload_libraries can spawn background workers which can perform any
> activity.  There were previous reports of corruption because of multi-xact
> being generated by such bgworkers during pg_upgrade, I'm pretty sure that there
> are some modules that create objects (automatic partitioning tools for
> instance).  It's also unclear to me what would happen if some writes are
> performed by such module at various points of the pg_upgrade process.  Couldn't
> that lead to either data loss or broken slot (as it couldn't stream changes
> from older major version)?
>

It won't be any bad than what can happen to tables. If we know that
such bgworkers can cause corruption if they do writes during the
upgrade, I don't think it is the job of this patch to prevent the
related scenarios. We can probably disallow the creation of new slots
during the binary upgrade but that also I am not sure. I guess it
would be better to document such hazards as a first step and then
probably write a patch to prevent WAL writes or something along those
lines.

-- 
With Regards,
Amit Kapila.



Commits

  1. Fix issues in binary_upgrade_logical_slot_has_caught_up().

  2. Fix a random failure in 003_logical_slots.pl.

  3. Fix a test in 003_logical_slots.

  4. Fix uninitialized slot array access during the upgrade.

  5. Fix the test 003_logical_slots.

  6. Commit b195e6d482 forgot to update meson.build.

  7. Use shorter file names in the upgrade logical slots test.

  8. Migrate logical slots to the new node during an upgrade.

  9. Flush logical slots to disk during a shutdown checkpoint if required.

  10. Prevent possibility of panics during shutdown checkpoint.