Re: repeated decoding of prepared transactions

Markus Wanner <markus.wanner@enterprisedb.com>

From: Markus Wanner <markus.wanner@enterprisedb.com>
To: Amit Kapila <amit.kapila16@gmail.com>, Ajin Cherian <itsajin@gmail.com>
Cc: Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>, Robert Haas <robertmhaas@gmail.com>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Simon Riggs <simon@enterprisedb.com>, Petr Jelinek <petr.jelinek@enterprisedb.com>
Date: 2021-02-10T08:10:19Z
Lists: pgsql-hackers
On 10.02.21 07:32, Amit Kapila wrote:
> On Wed, Feb 10, 2021 at 11:45 AM Ajin Cherian <itsajin@gmail.com> wrote:
>> But the other side of the problem is that ,without this, if the
>> prepared transaction is prior to a consistent snapshot when decoding
>> starts/restarts, then only the "commit prepared" is sent to downstream
>> (as seen in the test scenario I shared above), and downstream has to
>> error away the commit prepared because it does not have the
>> corresponding prepared transaction.
> 
> I think it is not only simple error handling, it is required for
> data-consistency. We need to send the transactions whose commits are
> encountered after a consistent snapshot is reached.

I'm with Ashutosh here.  If a replica is properly in sync, it knows 
about prepared transactions and all the gids of those.  Sending the 
transactional changes and the prepare again is inconsistent.

The point of a two-phase transaction is to have two phases.  An output 
plugin must have the chance of treating them as independent events. 
Once a PREPARE is confirmed, it must not be sent again.  Even if the 
transaction is still in-progress and its changes are not yet visible on 
the origin node.

Regards

Markus



Commits

  1. Add option to enable two_phase commits via pg_create_logical_replication_slot.

  2. Avoid repeated decoding of prepared transactions after a restart.

  3. Allow decoding at prepare time in ReorderBuffer.