Horton

Re: speed up a logical replica setup

Peter Eisentraut <peter@eisentraut.org>

View thread
From: Peter Eisentraut <peter@eisentraut.org>
To: vignesh C <vignesh21@gmail.com>, Euler Taveira <euler@eulerto.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Shlok Kyal <shlok.kyal.oss@gmail.com>, "kuroda.hayato@fujitsu.com" <kuroda.hayato@fujitsu.com>, Tomas Vondra <tomas.vondra@enterprisedb.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, Michael Paquier <michael@paquier.xyz>, Andres Freund <andres@anarazel.de>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Fabrízio de Royes Mello <fabriziomello@gmail.com>
Date: 2024-03-21T12:32:39Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. pg_createsubscriber: Remove obsolete comment

  2. pg_createsubscriber: Fix an unpredictable recovery wait time.

  3. Fix unstable test in 040_pg_createsubscriber.

  4. Fix the testcase introduced in commit 81d20fbf7a.

  5. Further weaken new pg_createsubscriber test on Windows.

  6. Temporarily(?) weaken new pg_createsubscriber test on Windows.

  7. Make pg_createsubscriber warn if publisher has two-phase commit enabled.

  8. Make pg_createsubscriber more wary about quoting connection parameters.

  9. pg_createsubscriber: Remove failover replication slots on subscriber

  10. pg_createsubscriber: Remove replication slot check on primary

  11. pg_createsubscriber: Only --recovery-timeout controls the end of recovery process

  12. pg_createsubscriber: creates a new logical replica from a standby server

  13. Add some const decorations

  14. Add option force_initdb to PostgreSQL::Test::Cluster:init()

  15. Remove MSVC scripts

On 21.03.24 12:35, vignesh C wrote:
> Here are a few suggestions:
> 1) I felt displaying the server log to the console is not a good idea,
> I prefer this to be logged. There were similar comments from
> Kuroda-san at [1], Peter at [2]. The number of lines will increase
> based on the log level set. If you don't want to use pg_upgrade style,
> how about exposing the log file option and logging it to the specified
> log file.

Let's leave that for the next version.  We need to wrap things up for 
this release.

> 2) Currently for publication, replication-slot and subscription, we
> will have to specify these options based on the number of databases.
> Here if we have 100 databases we will have to specify these options
> 100 times, it might not be user friendly. How about something like
> what Tomas had proposed at [3] and  Amit proposed at [4]. It will be
> better if the user has to just specify publication, replication slot
> and subscription options only one time.

Same.  Designing, implementing, discussing, and testing this cannot be 
done in the time remaining.

> +       /* Number of object names must match number of databases */
> +       if (num_pubs > 0 && num_pubs != num_dbs)
> +       {
> +               pg_log_error("wrong number of publication names");
> +               pg_log_error_hint("Number of publication names (%d)
> must match number of database names (%d).",
> +                                                 num_pubs, num_dbs);
> +               exit(1);
> +       }
> 
> 3) Can we have an option similar to dry-run which will display the
> configurations required in the primary and standby node something
> like:
> pg_createsubscriber -D data_N2/ -P "port=5431 user=postgres"  -p 9999
> -s /home/vignesh/postgres/inst/bin/ -U postgres -d db1 -d db2
> --suggest-config
> Suggested optimal configurations in the primary:
> --------------------------------------
> wallevel = logical
> max_replication_slots = 3
> max_wal_senders = 3
> ...
> Suggested optimal configurations in the standby:
> --------------------------------------
> max_replication_slots = 3
> max_wal_senders = 3
> ...

How would this be different from what --dry-run does now?