Re: repeated decoding of prepared transactions

Amit Kapila <amit.kapila16@gmail.com>

From: Amit Kapila <amit.kapila16@gmail.com>
To: Markus Wanner <markus.wanner@enterprisedb.com>
Cc: Ajin Cherian <itsajin@gmail.com>, Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>, Robert Haas <robertmhaas@gmail.com>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Simon Riggs <simon@enterprisedb.com>, Petr Jelinek <petr.jelinek@enterprisedb.com>
Date: 2021-02-10T10:10:37Z
Lists: pgsql-hackers
On Wed, Feb 10, 2021 at 1:40 PM Markus Wanner
<markus.wanner@enterprisedb.com> wrote:
>
> On 10.02.21 07:32, Amit Kapila wrote:
> > On Wed, Feb 10, 2021 at 11:45 AM Ajin Cherian <itsajin@gmail.com> wrote:
> >> But the other side of the problem is that ,without this, if the
> >> prepared transaction is prior to a consistent snapshot when decoding
> >> starts/restarts, then only the "commit prepared" is sent to downstream
> >> (as seen in the test scenario I shared above), and downstream has to
> >> error away the commit prepared because it does not have the
> >> corresponding prepared transaction.
> >
> > I think it is not only simple error handling, it is required for
> > data-consistency. We need to send the transactions whose commits are
> > encountered after a consistent snapshot is reached.
>
> I'm with Ashutosh here.  If a replica is properly in sync, it knows
> about prepared transactions and all the gids of those.  Sending the
> transactional changes and the prepare again is inconsistent.
>
> The point of a two-phase transaction is to have two phases.  An output
> plugin must have the chance of treating them as independent events.
>

I am not sure I understand what problem you are facing to deal with
this in the output plugin, it is explained in docs and Ajin also
pointed out the same. Ajin and I have explained to you the design
constraints on the publisher-side due to which we have done this way.
Do you have any better ideas to deal with this?

-- 
With Regards,
Amit Kapila.



Commits

  1. Add option to enable two_phase commits via pg_create_logical_replication_slot.

  2. Avoid repeated decoding of prepared transactions after a restart.

  3. Allow decoding at prepare time in ReorderBuffer.