Horton

Re: speed up a logical replica setup

Euler Taveira <euler@eulerto.com>

View thread
From: "Euler Taveira" <euler@eulerto.com>
To: "vignesh C" <vignesh21@gmail.com>
Cc: "Michael Paquier" <michael@paquier.xyz>, "Peter Eisentraut" <peter@eisentraut.org>, pgsql-hackers@lists.postgresql.org, "Amit Kapila" <amit.kapila16@gmail.com>, "Andres Freund" <andres@anarazel.de>, "Ashutosh Bapat" <ashutosh.bapat.oss@gmail.com>
Date: 2024-01-04T03:21:49Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. pg_createsubscriber: Remove obsolete comment

  2. pg_createsubscriber: Fix an unpredictable recovery wait time.

  3. Fix unstable test in 040_pg_createsubscriber.

  4. Fix the testcase introduced in commit 81d20fbf7a.

  5. Further weaken new pg_createsubscriber test on Windows.

  6. Temporarily(?) weaken new pg_createsubscriber test on Windows.

  7. Make pg_createsubscriber warn if publisher has two-phase commit enabled.

  8. Make pg_createsubscriber more wary about quoting connection parameters.

  9. pg_createsubscriber: Remove failover replication slots on subscriber

  10. pg_createsubscriber: Remove replication slot check on primary

  11. pg_createsubscriber: Only --recovery-timeout controls the end of recovery process

  12. pg_createsubscriber: creates a new logical replica from a standby server

  13. Add some const decorations

  14. Add option force_initdb to PostgreSQL::Test::Cluster:init()

  15. Remove MSVC scripts

On Mon, Jan 1, 2024, at 7:14 AM, vignesh C wrote:
> 1) This Assert can fail if source is shutdown:
> +static void
> +drop_replication_slot(PGconn *conn, LogicalRepInfo *dbinfo, const
> char *slot_name)
> +{
> +       PQExpBuffer str = createPQExpBuffer();
> +       PGresult   *res;
> +
> +       Assert(conn != NULL);

Oops. I'll remove it.

> 2) Should we have some checks to see if the max replication slot
> configuration is ok based on the number of slots that will be created,
> we have similar checks in upgrade replication slots in
> check_new_cluster_logical_replication_slots

That's a good idea.

> 3) Should we check if wal_level is set to logical, we have similar
> checks in upgrade replication slots in
> check_new_cluster_logical_replication_slots

That's a good idea.

> 4) The physical replication slot that was created will still be
> present in the primary node, I felt this should be removed.

My proposal is to remove it [1]. It'll be include in the next version.

> 5) I felt the target server should be started before completion of
> pg_subscriber:

Why? The initial version had an option to stop the subscriber. I decided to
remove the option and stop the subscriber by default mainly because (1) it is
an extra step to start the server (another point is that the WAL retention
doesn't happen due to additional (synchronized?) replication slots on
subscriber -- point 2). It was a conservative choice. If point 2 isn't an
issue, imo point 1 is no big deal.


[1] https://www.postgresql.org/message-id/e02a2c17-22e5-4ba6-b788-de696ab74f1e%40app.fastmail.com


--
Euler Taveira
EDB   https://www.enterprisedb.com/