Horton

Re: Replication slot stats misgivings

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: vignesh C <vignesh21@gmail.com>, Andres Freund <andres@anarazel.de>, pgsql-hackers <pgsql-hackers@postgresql.org>, Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
Date: 2021-04-28T06:25:33Z
Lists: pgsql-hackers 
On Wed, Apr 28, 2021 at 9:37 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Wed, Apr 28, 2021 at 12:29 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Tue, Apr 27, 2021 at 11:02 AM vignesh C <vignesh21@gmail.com> wrote:
> > >
> > > On Tue, Apr 27, 2021 at 9:48 AM vignesh C <vignesh21@gmail.com> wrote:
> > > >
> > >
> > > Attached patch has the changes to update statistics during
> > > spill/stream which prevents the statistics from being lost during
> > > interrupt.
> > >
> >
> >  void
> > -UpdateDecodingStats(LogicalDecodingContext *ctx)
> > +UpdateDecodingStats(ReorderBuffer *rb)
> >
> > I don't think you need to change this interface because
> > reorderbuffer->private_data points to LogicalDecodingContext. See
> > StartupDecodingContext.
>
> +1
>
> With this approach, we could still miss the totalTxns and totalBytes
> updates if the decoding a large but less than
> logical_decoding_work_mem is interrupted, right?
>

Right, but is there some simple way to avoid that? I see two
possibilities (a) store stats in ReplicationSlot and then send them at
ReplicationSlotRelease but that will lead to an increase in shared
memory usage and as per the discussion above, we don't want that, (b)
send intermediate stats after decoding say N changes but for that, we
need to additionally compute the size of each change which might add
some overhead.

I am not sure if any of these alternatives are a good idea. What do
you think? Do you have any other ideas for this?

-- 
With Regards,
Amit Kapila.

Commits

  1. Doc: Update logical decoding stats information.

  2. Fix tests for replication slots stats.

  3. Update replication statistics after every stream/spill.

  4. Fix the computation of slot stats for 'total_bytes'.

  5. Another try to fix the test case added by commit f5fc2f5b23.

  6. Use HTAB for replication slot statistics.

  7. Fix test case added by commit f5fc2f5b23.

  8. Add information of total data processed to replication slot stats.

  9. Use NameData datatype for slotname in stats.