Re: Minimal logical decoding on standbys

Amit Khandekar <amitdkhan.pg@gmail.com>

From: Amit Khandekar <amitdkhan.pg@gmail.com>

To: Andres Freund <andres@anarazel.de>

Cc: tushar <tushar.ahuja@enterprisedb.com>, Petr Jelinek <petr.jelinek@2ndquadrant.com>, Robert Haas <robertmhaas@gmail.com>, Craig Ringer <craig@2ndquadrant.com>, Petr Jelinek <petr@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>

Date: 2019-05-31T12:01:34Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Reduce the log level in 035_standby_logical_decoding.pl.
- 3034dc56ef4b 16.0 landed
035_standby_logical_decoding: Add missing waits for replication
- 57411c82ce86 16.0 landed
For cascading replication, wake physical and logical walsenders separately
- e101dfac3a53 16.0 landed
Handle logical slot conflicts on standby
- 26669757b6a7 16.0 landed
Support invalidating replication slots due to horizon and wal_level
- be87200efd93 16.0 landed
Prevent use of invalidated logical slot in CreateDecodingContext()
- 4397abd0a2af 16.0 landed
Replace replication slot's invalidated_at LSN with an enum
- 15f8203a5975 16.0 landed
Pass down table relation into more index relation functions
- 61b313e47eb9 16.0 landed
Assert only valid flag bits are passed to visibilitymap_set()
- a88a18b1250b 16.0 landed
Remove unused _bt_delitems_delete() argument.
- dc43492e46c7 14.0 cited
Add xl_btree_delete optimization.
- d2e5e20e5711 13.0 cited

On Fri, 31 May 2019 at 11:08, Amit Khandekar <amitdkhan.pg@gmail.com> wrote:
>
> On Thu, 30 May 2019 at 20:13, Andres Freund <andres@anarazel.de> wrote:
> >
> > Hi,
> >
> > On 2019-05-30 19:46:26 +0530, Amit Khandekar wrote:
> > > @@ -1042,7 +1042,8 @@ ReplicationSlotReserveWal(void)
> > >  if (!RecoveryInProgress() && SlotIsLogical(slot))
> > >  {
> > >     ....
> > >  }
> > >  else
> > >  {
> > > -   restart_lsn = GetRedoRecPtr();
> > > +   restart_lsn = SlotIsLogical(slot) ?
> > > +                        GetXLogReplayRecPtr(&ThisTimeLineID) : GetRedoRecPtr();
> > >
> > > But then when I do pg_create_logical_replication_slot(), it hangs in
> > > DecodingContextFindStartpoint(), waiting to find new records
> > > (XLogReadRecord).
> >
> > But just till the primary has logged the necessary WAL records? If you
> > just do CHECKPOINT; or such on the primary, it should succeed quickly?
>
> Yes, it waits until there is a commit record, or (just tried) until a
> checkpoint command.

Is XLOG_RUNNING_XACTS record essential for the logical decoding to
build a consistent snapshot ?
Since the restart_lsn is now ReplayRecPtr, there is no
XLOG_RUNNING_XACTS record, and so the snapshot state is not yet
SNAPBUILD_CONSISTENT. And so
DecodingContextFindStartpoint()=>DecodingContextReady() never returns
true, and hence DecodingContextFindStartpoint() goes in an infinite
loop, until it gets XLOG_RUNNING_XACTS.

-- 
Thanks,
-Amit Khandekar
EnterpriseDB Corporation
The Postgres Database Company