Horton

Re: Minimal logical decoding on standbys

tushar <tushar.ahuja@enterprisedb.com>

View thread
From: tushar <tushar.ahuja@enterprisedb.com>
To: Andres Freund <andres@anarazel.de>
Cc: Petr Jelinek <petr.jelinek@2ndquadrant.com>, Robert Haas <robertmhaas@gmail.com>, Craig Ringer <craig@2ndquadrant.com>, Petr Jelinek <petr@2ndquadrant.com>, pgsql-hackers@postgresql.org
Date: 2019-03-04T11:24:32Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Reduce the log level in 035_standby_logical_decoding.pl.

  2. 035_standby_logical_decoding: Add missing waits for replication

  3. For cascading replication, wake physical and logical walsenders separately

  4. Handle logical slot conflicts on standby

  5. Support invalidating replication slots due to horizon and wal_level

  6. Prevent use of invalidated logical slot in CreateDecodingContext()

  7. Replace replication slot's invalidated_at LSN with an enum

  8. Pass down table relation into more index relation functions

  9. Assert only valid flag bits are passed to visibilitymap_set()

  10. Remove unused _bt_delitems_delete() argument.

  11. Add xl_btree_delete optimization.

On 03/01/2019 11:16 PM, Andres Freund wrote:
> So, if I understand correctly you do*not*  have a phyiscal replication
> slot for this standby? For the feature to work reliably that needs to
> exist, and you need to have hot_standby_feedback enabled. Does having
> that fix the issue?

Ok, This time around  - I performed like this -

.)Master cluster (set wal_level=logical and hot_standby_feedback=on in 
postgresql.conf) , start the server and create a physical replication slot

postgres=# SELECT * FROM 
pg_create_physical_replication_slot('decoding_standby');
     slot_name     | lsn
------------------+-----
  decoding_standby |
(1 row)

.)Perform pg_basebackup using --slot=decoding_standby  with option -R . 
modify port=5555 , start the server

.)Connect to slave and create a logical replication slot

postgres=# create table t(n int);
ERROR:  cannot execute CREATE TABLE in a read-only transaction
postgres=#

postgres=# SELECT * FROM 
pg_create_logical_replication_slot('standby_slot', 'test_decoding');
   slot_name   |    lsn
--------------+-----------
  standby_slot | 0/2000060
(1 row)

run pgbench (./pgbench -i -s 10 postgres) against  master and 
simultaneously- start  pg_recvlogical  , provide port=5555 ( slave 
cluster)  and specify slot=standby_slot
./pg_recvlogical -d postgres  -p 5555 -s 1 -F 1  -v --slot=standby_slot  
--start -f -


[centos@centos-cpula bin]$ ./pg_recvlogical -d postgres  -p 5555 -s 1 -F 
1  -v --slot=standby_slot  --start -f -
pg_recvlogical: starting log streaming at 0/0 (slot standby_slot)
pg_recvlogical: streaming initiated
pg_recvlogical: confirming write up to 0/0, flush to 0/0 (slot standby_slot)
pg_recvlogical: confirming write up to 0/30194E8, flush to 0/30194E8 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3019590, flush to 0/3019590 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3019590, flush to 0/3019590 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3019590, flush to 0/3019590 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3019590, flush to 0/3019590 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3019590, flush to 0/3019590 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3019590, flush to 0/3019590 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3019590, flush to 0/3019590 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3019590, flush to 0/3019590 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3019590, flush to 0/3019590 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/301D558, flush to 0/301D558 
(slot standby_slot)
BEGIN 476
COMMIT 476
pg_recvlogical: confirming write up to 0/3034B40, flush to 0/3034B40 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3034B40, flush to 0/3034B40 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3034B40, flush to 0/3034B40 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3034B40, flush to 0/3034B40 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3034B40, flush to 0/3034B40 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3034B40, flush to 0/3034B40 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3034B40, flush to 0/3034B40 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3034B40, flush to 0/3034B40 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3034B40, flush to 0/3034B40 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3034B40, flush to 0/3034B40 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3034B40, flush to 0/3034B40 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3034B40, flush to 0/3034B40 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3034B40, flush to 0/3034B40 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3034B40, flush to 0/3034B40 
(slot standby_slot)
pg_recvlogical: confirming write up to 0/3034B40, flush to 0/3034B40 
(slot standby_slot)
BEGIN 477
COMMIT 477

If we do the same for the logical replication slot which created on 
Master cluster

[centos@centos-cpula bin]$ ./pg_recvlogical -d postgres  -s 1 -F 1 -v 
--slot=master_slot  --start -f -
pg_recvlogical: starting log streaming at 0/0 (slot master_slot)
pg_recvlogical: streaming initiated
pg_recvlogical: confirming write up to 0/0, flush to 0/0 (slot master_slot)
table public.pgbench_accounts: INSERT: aid[integer]:65057 bid[integer]:1 
abalance[integer]:0 filler[character]:' '
table public.pgbench_accounts: INSERT: aid[integer]:65058 bid[integer]:1 
abalance[integer]:0 filler[character]:' '
table public.pgbench_accounts: INSERT: aid[integer]:65059 bid[integer]:1 
abalance[integer]:0 filler[character]:' '
table public.pgbench_accounts: INSERT: aid[integer]:65060 bid[integer]:1 
abalance[integer]:0 filler[character]:' '
table public.pgbench_accounts: INSERT: aid[integer]:65061 bid[integer]:1 
abalance[integer]:0 filler[character]:' '
table public.pgbench_accounts: INSERT: aid[integer]:65062 bid[integer]:1 
abalance[integer]:0 filler[character]:' '
table public.pgbench_accounts: INSERT: aid[integer]:65063 bid[integer]:1 
abalance[integer]:0 filler[character]:' '
table public.pgbench_accounts: INSERT: aid[integer]:65064 bid[integer]:1 
abalance[integer]:0 filler[character]:' '

-- 
regards,tushar
EnterpriseDB  https://www.enterprisedb.com/
The Enterprise PostgreSQL Company