Horton

Re: speed up a logical replica setup

Peter Eisentraut <peter@eisentraut.org>

View thread
From: Peter Eisentraut <peter@eisentraut.org>
To: vignesh C <vignesh21@gmail.com>, Euler Taveira <euler@eulerto.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Shlok Kyal <shlok.kyal.oss@gmail.com>, "kuroda.hayato@fujitsu.com" <kuroda.hayato@fujitsu.com>, Tomas Vondra <tomas.vondra@enterprisedb.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, Michael Paquier <michael@paquier.xyz>, Andres Freund <andres@anarazel.de>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Fabrízio de Royes Mello <fabriziomello@gmail.com>
Date: 2024-03-18T11:06:30Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. pg_createsubscriber: Remove obsolete comment

  2. pg_createsubscriber: Fix an unpredictable recovery wait time.

  3. Fix unstable test in 040_pg_createsubscriber.

  4. Fix the testcase introduced in commit 81d20fbf7a.

  5. Further weaken new pg_createsubscriber test on Windows.

  6. Temporarily(?) weaken new pg_createsubscriber test on Windows.

  7. Make pg_createsubscriber warn if publisher has two-phase commit enabled.

  8. Make pg_createsubscriber more wary about quoting connection parameters.

  9. pg_createsubscriber: Remove failover replication slots on subscriber

  10. pg_createsubscriber: Remove replication slot check on primary

  11. pg_createsubscriber: Only --recovery-timeout controls the end of recovery process

  12. pg_createsubscriber: creates a new logical replica from a standby server

  13. Add some const decorations

  14. Add option force_initdb to PostgreSQL::Test::Cluster:init()

  15. Remove MSVC scripts

On 18.03.24 08:18, vignesh C wrote:
> 1) Maximum size of the object name is 64, we can have a check so that
> we don't specify more than the maximum allowed length:
> + case 3:
> + if (!simple_string_list_member(&opt.replslot_names, optarg))
> + {
> + simple_string_list_append(&opt.replslot_names, optarg);
> + num_replslots++;
> + }
> + else
> + {
> + pg_log_error("duplicate replication slot \"%s\"", optarg);
> + exit(1);
> + }
> + break;
> 
> If we allow something like this:
>   ./pg_createsubscriber -U postgres -D data_N2/ -P "port=5431
> user=postgres"  -p 5432 -s /home/vignesh/postgres/inst/bin/ -d db1 -d
> db2 -d db3 --replication-slot="testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttes1"
> --replication-slot="testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttes2"
> --replication-slot="testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttes3"
> In this case creation of replication slot will fail:
> pg_createsubscriber: error: could not create replication slot
> "testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttes" on
> database "db2": ERROR:  replication slot
> "testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttes"
> already exists

I think this is fine.  The server can check whether the names it is 
given are of the right size.  We don't need to check it again in the client.

> 2) Similarly here too:
> + case 4:
> + if (!simple_string_list_member(&opt.sub_names, optarg))
> + {
> + simple_string_list_append(&opt.sub_names, optarg);
> + num_subs++;
> + }
> + else
> + {
> + pg_log_error("duplicate subscription \"%s\"", optarg);
> + exit(1);
> + }
> + break;
> 
> If we allow something like this:
> ./pg_createsubscriber -U postgres -D data_N2/ -P "port=5431
> user=postgres"  -p 5432 -s /home/vignesh/postgres/inst/bin/ -d db1 -d
> db2 -d db3 --subscription=testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttes1
> --subscription=testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttes2
> --subscription=testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttes3
> 
> Subscriptions will be created with the same name and later there will
> be a problem when setting replication progress as there will be
> multiple subscriptions with the same name.

Could you clarify this problem?