Horton

RE: speed up a logical replica setup

Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>

View thread
From: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
To: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, 'Euler Taveira' <euler@eulerto.com>
Cc: "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, vignesh C <vignesh21@gmail.com>, Michael Paquier <michael@paquier.xyz>, Peter Eisentraut <peter@eisentraut.org>, Andres Freund <andres@anarazel.de>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Shlok Kyal <shlok.kyal.oss@gmail.com>, Fabrízio de Royes Mello <fabriziomello@gmail.com>
Date: 2024-02-19T05:45:38Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. pg_createsubscriber: Remove obsolete comment

  2. pg_createsubscriber: Fix an unpredictable recovery wait time.

  3. Fix unstable test in 040_pg_createsubscriber.

  4. Fix the testcase introduced in commit 81d20fbf7a.

  5. Further weaken new pg_createsubscriber test on Windows.

  6. Temporarily(?) weaken new pg_createsubscriber test on Windows.

  7. Make pg_createsubscriber warn if publisher has two-phase commit enabled.

  8. Make pg_createsubscriber more wary about quoting connection parameters.

  9. pg_createsubscriber: Remove failover replication slots on subscriber

  10. pg_createsubscriber: Remove replication slot check on primary

  11. pg_createsubscriber: Only --recovery-timeout controls the end of recovery process

  12. pg_createsubscriber: creates a new logical replica from a standby server

  13. Add some const decorations

  14. Add option force_initdb to PostgreSQL::Test::Cluster:init()

  15. Remove MSVC scripts

Attachments

Dear hackers,

> Since it may be useful, I will post top-up patch on Monday, if there are no
> updating.

And here are top-up patches. Feel free to check and include.

v22-0001: Same as v21-0001.
=== rebased patches ===
v22-0002: Update docs per recent changes. Same as v20-0002.
v22-0003: Add check versions of the target. Extracted from v20-0003.
v22-0004: Remove -S option. Mostly same as v20-0009, but commit massage was
          slightly changed.
=== Newbie ===
V22-0005: Addressed my comments which seems to be trivial[1].
          Comments #1, 3, 4, 8, 10, 14, 17 were addressed here.
v22-0006: Consider the scenario when commands are failed after the recovery.
          drop_subscription() is removed and some messages are added per [2].
V22-0007: Revise server_is_in_recovery() per [1]. Comments #5, 6, 7, were addressed here.
V22-0008: Fix a strange report when physical_primary_slot is null. Per comment #9 [1].
V22-0009: Prohibit reuse publications when it has already existed. Per comments #11 and 12 [1].
V22-0010: Avoid to call PQclear()/PQfinish()/pg_free() if the process exits soon. Per comment #15 [1].
V22-0011: Update testcode. Per comments #17- [1].

I did not handle below points because I have unclear points.

a. 
This patch set cannot detect the disconnection between the target (standby) and
source (primary) during the catch up. Because the connection status must be gotten
at the same time (=in the same query) with the recovery status, but now it is now an
independed function (server_is_in_recovery()).

b.
This patch set cannot detect the inconsistency reported by Shubham [3]. I could not
come up with solutions without removing -P...

[1]: https://www.postgresql.org/message-id/TYCPR01MB12077756323B79042F29DDAEDF54C2%40TYCPR01MB12077.jpnprd01.prod.outlook.com
[2]: https://www.postgresql.org/message-id/TYCPR01MB12077E98F930C3DE6BD304D0DF54C2%40TYCPR01MB12077.jpnprd01.prod.outlook.com
[3]: https://www.postgresql.org/message-id/CAHv8Rj%2B5mzK9Jt%2B7ECogJzfm5czvDCCd5jO1_rCx0bTEYpBE5g%40mail.gmail.com

Best Regards,
Hayato Kuroda
FUJITSU LIMITED
https://www.fujitsu.com/