Re: [PoC] pg_upgrade: allow to upgrade publisher node

Amit Kapila <amit.kapila16@gmail.com>

From: Amit Kapila <amit.kapila16@gmail.com>
To: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
Cc: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Bruce Momjian <bruce@momjian.us>, Julien Rouhaud <rjuju123@gmail.com>, vignesh C <vignesh21@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>
Date: 2023-09-08T11:23:32Z
Lists: pgsql-hackers
On Fri, Sep 8, 2023 at 2:12 PM Zhijie Hou (Fujitsu)
<houzj.fnst@fujitsu.com> wrote:
>
> 2.
>
> +       if (nslots_on_new)
> +       {
> +               if (nslots_on_new == 1)
> +                       pg_fatal("New cluster must not have logical replication slots but found a slot.");
> +               else
> +                       pg_fatal("New cluster must not have logical replication slots but found %d slots.",
> +                                        nslots_on_new);
>
> We could try ngettext() here:
>                 pg_log_warning(ngettext("New cluster must not have logical replication slots but found %d slot.",
>                                                                 "New cluster must not have logical replication slots but found %d slots",
>                                                                 nslots_on_new)
>

Will using pg_log_warning suffice for the purpose of exiting the
upgrade process? I don't think the intention here is to continue after
finding such a case.

>
> 4.
>
> @@ -610,6 +724,12 @@ free_db_and_rel_infos(DbInfoArr *db_arr)
>         {
>                 free_rel_infos(&db_arr->dbs[dbnum].rel_arr);
>                 pg_free(db_arr->dbs[dbnum].db_name);
> +
> +               /*
> +                * Logical replication slots must not exist on the new cluster before
> +                * create_logical_replication_slots().
> +                */
> +               Assert(db_arr->dbs[dbnum].slot_arr.nslots == 0);
>
>
> I think the assert is not necessary, as the patch will check the new cluster's
> slots in another function. Besides, this function is not only used for new
> cluster, but the comment only mentioned the new cluster which seems a bit
> inconsistent. So, how about removing it ?
>

Yeah, I also find it odd.

> 5.
>                          (cluster == &new_cluster) ?
> -                        " -c synchronous_commit=off -c fsync=off -c full_page_writes=off" : "",
> +                        " -c synchronous_commit=off -c fsync=off -c full_page_writes=off" :
> +                        " -c max_slot_wal_keep_size=-1",
>
> I think we need to set max_slot_wal_keep_size on new cluster as well, otherwise
> it's possible that the new created slots get invalidated during upgrade, what
> do you think ?
>

I also think that would be better.

> 6.
>
> +       bool            is_lost;                /* Is the slot in 'lost'? */
> +} LogicalSlotInfo;
>
> Would it be better to use 'invalidated',
>

Or how about simply 'invalid'?

A few other points:
1.
  ntups = PQntuples(res);
- dbinfos = (DbInfo *) pg_malloc(sizeof(DbInfo) * ntups);
+ dbinfos = (DbInfo *) pg_malloc0(sizeof(DbInfo) * ntups);

Can we write a comment to say why we need zero memory here?

 2. Why get_old_cluster_logical_slot_infos() need to use
pg_malloc_array whereas for similar stuff get_rel_infos() use
pg_malloc()?

-- 
With Regards,
Amit Kapila.



Commits

  1. Fix issues in binary_upgrade_logical_slot_has_caught_up().

  2. Fix a random failure in 003_logical_slots.pl.

  3. Fix a test in 003_logical_slots.

  4. Fix uninitialized slot array access during the upgrade.

  5. Fix the test 003_logical_slots.

  6. Commit b195e6d482 forgot to update meson.build.

  7. Use shorter file names in the upgrade logical slots test.

  8. Migrate logical slots to the new node during an upgrade.

  9. Flush logical slots to disk during a shutdown checkpoint if required.

  10. Prevent possibility of panics during shutdown checkpoint.