Horton

Re: logical decoding and replication of sequences, take 2

Amit Kapila <amit.kapila16@gmail.com>

View thread
From: Amit Kapila <amit.kapila16@gmail.com>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Masahiko Sawada <sawada.mshk@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, Dilip Kumar <dilipbalaut@gmail.com>
Date: 2023-12-06T08:56:21Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Migrate logical slots to the new node during an upgrade.

  2. Make test_decoding ddl.out shorter

  3. Fix snapshot handling in logicalmsg_decode

  4. doc: Adjust a few more references to "postmaster"

  5. Revert "Logical decoding of sequences"

On Tue, Dec 5, 2023 at 10:23 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
> On 12/5/23 13:17, Amit Kapila wrote:
>
> > (b) for transactional
> > cases, we see overhead due to traversing all the top-level txns and
> > check the hash table for each one to find whether change is
> > transactional.
> >
>
> Not really, no. As I explained in my preceding e-mail, this check makes
> almost no difference - I did expect it to matter, but it doesn't. And I
> was a bit disappointed the global hash table didn't move the needle.
>
> Most of the time is spent in
>
>     78.81%     0.00%  postgres  postgres  [.] DecodeCommit (inlined)
>       |
>       ---DecodeCommit (inlined)
>          |
>          |--72.65%--SnapBuildCommitTxn
>          |     |
>          |      --72.61%--SnapBuildBuildSnapshot
>          |            |
>          |             --72.09%--pg_qsort
>          |                    |
>          |                    |--66.24%--pg_qsort
>          |                    |          |
>
> And there's almost no difference between master and build with sequence
> decoding - see the attached diff-alter-sequence.perf, comparing the two
> branches (perf diff -c delta-abs).
>

I think in this the commit time predominates which hides the overhead.
We didn't investigate in detail if that can be improved but if we see
a similar case of abort [1], it shows the overhead of
ReorderBufferSequenceIsTransactional(). I understand that aborts won't
be frequent and it is sort of unrealistic test but still helps to show
that there is overhead in ReorderBufferSequenceIsTransactional(). Now,
I am not sure if we can ignore that case because theoretically, the
overhead can increase based on the number of top-level transactions.

[1]: https://www.postgresql.org/message-id/TY3PR01MB9889D457278B254CA87D1325F581A%40TY3PR01MB9889.jpnprd01.prod.outlook.com

-- 
With Regards,
Amit Kapila.