Re: repeated decoding of prepared transactions

Andres Freund <andres@anarazel.de>

From: Andres Freund <andres@anarazel.de>
To: Ajin Cherian <itsajin@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Markus Wanner <markus.wanner@enterprisedb.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2021-02-20T20:13:17Z
Lists: pgsql-hackers
Hi,

On 2021-02-19 13:50:52 +1100, Ajin Cherian wrote:
> From 129947ab2d0ba223862ed1c87be0f96b51645ba0 Mon Sep 17 00:00:00 2001
> From: Ajin Cherian <ajinc@fast.au.fujitsu.com>
> Date: Thu, 18 Feb 2021 20:18:16 -0500
> Subject: [PATCH] Don't allow repeated decoding of prepared transactions.
> 
> Enforce full snapshot while decoding with two-phase enabled. This
> allows the decoder to differentiate between prepared transaction that
> were sent prior to restart and prepared transactions that were not sent
> because they were prior to consistent snapshot.

Isn't this an *extremely* expensive solution? Maintaining a full
snapshot is pretty darn expensive - so expensive that it's repeatedly
been a problem even just for building the initial snapshot (to the point
of being inable to do so). And that's typically a comparatively rare
operation, not something continual - but what you're proposing is a cost
paid during ongoing replication.

Greetings,

Andres Freund



Commits

  1. Add option to enable two_phase commits via pg_create_logical_replication_slot.

  2. Avoid repeated decoding of prepared transactions after a restart.

  3. Allow decoding at prepare time in ReorderBuffer.