Horton

Re: Replication slot stats misgivings

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: vignesh C <vignesh21@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Masahiko Sawada <sawada.mshk@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
Date: 2021-04-16T09:46:18Z
Lists: pgsql-hackers 
On Mon, Apr 12, 2021 at 2:57 PM vignesh C <vignesh21@gmail.com> wrote:
>
> On Sat, Mar 20, 2021 at 9:26 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Sat, Mar 20, 2021 at 12:22 AM Andres Freund <andres@anarazel.de> wrote:
> > >
> > > And then more generally about the feature:
> > > - If a slot was used to stream out a large amount of changes (say an
> > >   initial data load), but then replication is interrupted before the
> > >   transaction is committed/aborted, stream_bytes will not reflect the
> > >   many gigabytes of data we may have sent.
> > >
> >
> > We can probably update the stats each time we spilled or streamed the
> > transaction data but it was not clear at that stage whether or how
> > much it will be useful.
> >
>
> I felt we can update the replication slot statistics data each time we
> spill/stream the transaction data instead of accumulating the
> statistics and updating at the end. I have tried this in the attached
> patch and the statistics data were getting updated.
> Thoughts?
>

Did you check if we can update the stats when we release the slot as
discussed above? I am not sure if it is easy to do at the time of slot
release because this information might not be accessible there and in
some cases, we might have already released the decoding
context/reorderbuffer where this information is stored. It might be
okay to update this when we stream or spill but let's see if we can do
it easily at the time of slot release.

-- 
With Regards,
Amit Kapila.

Commits

  1. Doc: Update logical decoding stats information.

  2. Fix tests for replication slots stats.

  3. Update replication statistics after every stream/spill.

  4. Fix the computation of slot stats for 'total_bytes'.

  5. Another try to fix the test case added by commit f5fc2f5b23.

  6. Use HTAB for replication slot statistics.

  7. Fix test case added by commit f5fc2f5b23.

  8. Add information of total data processed to replication slot stats.

  9. Use NameData datatype for slotname in stats.