RE: [PoC] pg_upgrade: allow to upgrade publisher node

Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>

From: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
To: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Cc: Julien Rouhaud <rjuju123@gmail.com>, vignesh C <vignesh21@gmail.com>, 'Amit Kapila' <amit.kapila16@gmail.com>, 'Masahiko Sawada' <sawada.mshk@gmail.com>, "'Jonathan S. Katz'" <jkatz@postgresql.org>
Date: 2023-08-10T15:02:43Z
Lists: pgsql-hackers

Attachments

Dear hackers,

Based on recent discussions, I updated the patch set. I did not reply one by one
because there are many posts, but thank you for giving many suggestion!

Followings shows what I changed.

1.
This feature is now enabled by default. Instead "--exclude-logical-replication-slots"
was added. (Per suggestions like [1])

2.
Pg_upgrade raises ERROR when some slots are 'WALAVAIL_REMOVED'. (Per discussion[2])

3.
Slots which are 'WALAVAIL_UNRESERVED' are dumped and restored. (Per consideration[3])

4.
Combination --logical-replication-slots-only and other --only options was
prohibit again. (Per suggestion[4]) Currently --data-only and --schema-only
could not be used together, so I followed the same style. Additionally, it's not
easy for user to predict the behavior if specifying many --only command.

5. 
Fixed some bugs related with combinations of options. E.g., v18 did not allow to
use "--create", but now it could use same time. This was because information
of role did not get from node while doing slot dump.

6.
The ordering of patches was changed. The patch "Always persist to disk..."
became 0001. (Per suggestion [4])

7.
Functions for checking were changed (per [5]). Currently WALs between
confirmed_lsn and current location is scanned and confirmed. The requirements
are little hacky:

* The first record after the confirmed_lsn must be SHUTDOWN_CHECKPOINT
* Other records till current position must be either RUNNING_XACT,
  CHECKPOINT_ONLINE or XLOG_FPI_FOR_HINT.

In the checking function (validate_wal_record_types_after), WALs are read
repeatedly and confirmed its type. v18 required to change the version number
for pg_walinspect, it is not needed anymore.


[1]: https://www.postgresql.org/message-id/ad83b9f2-ced3-c51c-342a-cc281ff562fc%40postgresql.org
[2]: https://www.postgresql.org/message-id/CAA4eK1%2B8btsYhNQvw6QJ4iTw1wFhkFXXABT%3DED1eHFvtekRanQ%40mail.gmail.com
[3]: https://www.postgresql.org/message-id/TYAPR01MB5866FD3F7992A46D0457F0E6F50BA%40TYAPR01MB5866.jpnprd01.prod.outlook.com
[4]: https://www.postgresql.org/message-id/CAA4eK1%2BCD82Kssy%2BiqpETPKYUh9AmNORF%2B3iGfNXgxKxqL3T6g%40mail.gmail.com
[5]: https://www.postgresql.org/message-id/CAD21AoC4D4wYTcLM8T-rAv%3DpO5kS6ffcVD1e7h4eFERT4%2BfwQQ%40mail.gmail.com

Best Regards,
Hayato Kuroda
FUJITSU LIMITED

Commits

  1. Fix issues in binary_upgrade_logical_slot_has_caught_up().

  2. Fix a random failure in 003_logical_slots.pl.

  3. Fix a test in 003_logical_slots.

  4. Fix uninitialized slot array access during the upgrade.

  5. Fix the test 003_logical_slots.

  6. Commit b195e6d482 forgot to update meson.build.

  7. Use shorter file names in the upgrade logical slots test.

  8. Migrate logical slots to the new node during an upgrade.

  9. Flush logical slots to disk during a shutdown checkpoint if required.

  10. Prevent possibility of panics during shutdown checkpoint.