Horton

Re: Replication slot stats misgivings

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Andres Freund <andres@anarazel.de>, PostgreSQL-development <pgsql-hackers@postgresql.org>, Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
Date: 2021-05-12T02:23:04Z
Lists: pgsql-hackers 
On Wed, May 12, 2021 at 4:00 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Wed, May 12, 2021 at 6:32 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >
> > Amit Kapila <amit.kapila16@gmail.com> writes:
> > > I have closed this open item.
> >
> > That seems a little premature, considering that the
> > contrib/test_decoding/sql/stats.sql test case is still failing regularly.
>
> Thank you for reporting.
>
> Ugh, since by commit 592f00f8de we send slot stats every after
> spil/stream it’s possible that we report slot stats that have non-zero
> counters for spill_bytes/txns and zeroes for total_bytes/txns. It
> seems to me it’s legitimate that the slot stats view shows non-zero
> values for spill_bytes/txns and zero values for total_bytes/txns
> during decoding a large transaction. So I think we can fix the test
> script so that it checks only spill_bytes/txns when checking spilled
> transactions.
>

Your analysis and fix look correct to me. I'll test and push your
patch if I don't see any problem with it.

> For the record, during streaming transactions, IIUC this kind of thing
> doesn’t happen since we update both total_bytes/txns and
> stream_bytes/txns before reporting slot stats.
>

Right, because during streaming, we send the data to the decoding plugin.

-- 
With Regards,
Amit Kapila.

Commits

  1. Doc: Update logical decoding stats information.

  2. Fix tests for replication slots stats.

  3. Update replication statistics after every stream/spill.

  4. Fix the computation of slot stats for 'total_bytes'.

  5. Another try to fix the test case added by commit f5fc2f5b23.

  6. Use HTAB for replication slot statistics.

  7. Fix test case added by commit f5fc2f5b23.

  8. Add information of total data processed to replication slot stats.

  9. Use NameData datatype for slotname in stats.