Re: [PoC] pg_upgrade: allow to upgrade publisher node

vignesh C <vignesh21@gmail.com>

From: vignesh C <vignesh21@gmail.com>
To: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
Cc: Peter Smith <smithpb2250@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Andres Freund <andres@anarazel.de>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, Dilip Kumar <dilipbalaut@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Bruce Momjian <bruce@momjian.us>, Julien Rouhaud <rjuju123@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: 2023-10-20T03:19:08Z
Lists: pgsql-hackers
On Thu, 19 Oct 2023 at 16:14, Hayato Kuroda (Fujitsu)
<kuroda.hayato@fujitsu.com> wrote:
>
> Dear Vignesh,
>
> Thanks for reviewing! New patch can be available in [1].
>
> >
> > Few comments:
> > 1) We will be able to override the value of max_slot_wal_keep_size by
> > using --new-options like '--new-options  "-c
> > max_slot_wal_keep_size=val"':
> > +       /*
> > +        * Use max_slot_wal_keep_size as -1 to prevent the WAL removal by the
> > +        * checkpointer process.  If WALs required by logical replication slots
> > +        * are removed, the slots are unusable.  This setting prevents the
> > +        * invalidation of slots during the upgrade. We set this option when
> > +        * cluster is PG17 or later because logical replication slots
> > can only be
> > +        * migrated since then. Besides, max_slot_wal_keep_size is
> > added in PG13.
> > +        */
> > +       if (GET_MAJOR_VERSION(cluster->major_version) >= 1700)
> > +               appendPQExpBufferStr(&pgoptions, " -c
> > max_slot_wal_keep_size=-1");
> >
> > Should there be a check to throw an error if this option is specified
> > or do we need some documentation that this option should not be
> > specified?
>
> Hmm, I don't think we have to add checks. Other settings, like synchronous_commit
> and fsync, can be also overwritten, but pg_upgrade has never checked. Therefore,
> it's user's responsibility to not set max_slot_wal_keep_size to a dangerous
> value.
>
> > 2) Because we are able to override max_slot_wal_keep_size there is a
> > chance of slot getting invalidated and Assert being hit:
> > +               /*
> > +                * The logical replication slots shouldn't be invalidated as
> > +                * max_slot_wal_keep_size GUC is set to -1 during the
> > upgrade.
> > +                *
> > +                * The following is just a sanity check.
> > +                */
> > +               if (*invalidated && SlotIsLogical(s) && IsBinaryUpgrade)
> > +               {
> > +                       Assert(max_slot_wal_keep_size_mb == -1);
> > +                       elog(ERROR, "replication slots must not be
> > invalidated during the upgrade");
> > +               }
>
> Hmm, so how about removing an assert and changing the error message more
> appropriate? I still think it seldom occurs.

As this scenario can occur by overriding max_slot_wal_keep_size, it is
better to remove the Assert.

Regards,
Vignesh



Commits

  1. Fix issues in binary_upgrade_logical_slot_has_caught_up().

  2. Fix a random failure in 003_logical_slots.pl.

  3. Fix a test in 003_logical_slots.

  4. Fix uninitialized slot array access during the upgrade.

  5. Fix the test 003_logical_slots.

  6. Commit b195e6d482 forgot to update meson.build.

  7. Use shorter file names in the upgrade logical slots test.

  8. Migrate logical slots to the new node during an upgrade.

  9. Flush logical slots to disk during a shutdown checkpoint if required.

  10. Prevent possibility of panics during shutdown checkpoint.