Horton

RE: [PoC] pg_upgrade: allow to upgrade publisher node

Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>

View thread View on postgr.es
From: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
To: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Peter Smith <smithpb2250@gmail.com>, 'Bharath Rupireddy' <bharath.rupireddyforpostgres@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Bruce Momjian <bruce@momjian.us>, Julien Rouhaud <rjuju123@gmail.com>, vignesh C <vignesh21@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>
Date: 2023-09-26T04:48:40Z
Lists: pgsql-hackers 
On Monday, September 25, 2023 7:01 PM Kuroda, Hayato/黒田 隼人 <kuroda.hayato@fujitsu.com> wrote:
> To: 'Bharath Rupireddy' <bharath.rupireddyforpostgres@gmail.com>
> Cc: Amit Kapila <amit.kapila16@gmail.com>; Dilip Kumar
> >
> > 5. In continuation to the above comment:
> >
> > Why can't this logic be something like - if there's any WAL record
> > seen after a slot's confirmed flush LSN is of type generated by WAL
> > resource manager having the rm_decode function defined, then the slot
> > can't be upgraded.
> 
> Thank you for giving new approach! We have never seen the approach before,
> but at least XLOG and HEAP2 rmgr have a decode function. So that
> XLOG_CHECKPOINT_SHUTDOWN, XLOG_CHECKPOINT_ONLINE, and
> XLOG_HEAP2_PRUNE cannot be ignored the approach, seems not appropriate.
> If you have another approach, I'm very happy if you post.

Another idea around decoding is to check if there is any decoding output for
the WAL records.

Like we can create a temp slot and use test_decoding to decode the WAL from the
confirmed_flush_lsn among existing logical replication slots. And if there is
any output from the output plugin, then we consider WAL has not been consumed
yet.

But this means we need to ignore some of the WALs like XLOG_XACT_INVALIDATIONS
which won't be decoded into the output. Also, this approach could be costly as
it needs to do the extra decoding and output, and we need to assume that "all the
WAL records including custom records will be decoded and output if they need to
be consumed" .

So it may not be better, but just share it for reference.

Best Regards,
Hou zj

Commits

  1. Fix issues in binary_upgrade_logical_slot_has_caught_up().

  2. Fix a random failure in 003_logical_slots.pl.

  3. Fix a test in 003_logical_slots.

  4. Fix uninitialized slot array access during the upgrade.

  5. Fix the test 003_logical_slots.

  6. Commit b195e6d482 forgot to update meson.build.

  7. Use shorter file names in the upgrade logical slots test.

  8. Migrate logical slots to the new node during an upgrade.

  9. Flush logical slots to disk during a shutdown checkpoint if required.

  10. Prevent possibility of panics during shutdown checkpoint.