Horton

Re: [BUG]Update Toast data failure in logical replication

Dilip Kumar <dilipbalaut@gmail.com>

View thread
From: Dilip Kumar <dilipbalaut@gmail.com>
To: Ajin Cherian <itsajin@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Petr Jelinek <petr.jelinek@enterprisedb.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, Tomas Vondra <tomas.vondra@enterprisedb.com>, Andres Freund <andres@anarazel.de>, Kuntal Ghosh <kuntalghosh.2007@gmail.com>, "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2021-09-08T06:28:17Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. WAL log unchanged toasted replica identity key attributes.

  2. Add new wal_level, logical, sufficient for logical decoding.

  3. Add the notion of REPLICA IDENTITY for a table.

On Wed, Sep 8, 2021 at 11:26 AM Ajin Cherian <itsajin@gmail.com> wrote:

> On Wed, Aug 11, 2021 at 10:45 PM Dilip Kumar <dilipbalaut@gmail.com>
> wrote:
>
> > Yeah we can avoid that by detecting any toasted replica identity key
> > in HeapDetermineModifiedColumns, check the attached patch.
> >
>
> I had a second look at this, and I just had a small doubt. Since the
> convention is that for UPDATES, the old tuple/key is written to
> WAL only if the one of the columns in the key has changed as part of
> the update, and we are breaking that convention with this patch by
> also including
> the old key if it is toasted and is stored in disk even if it is not
> changed.
> Why do we not include the detoasted key as part of the new tuple
> rather than the old tuple? Then we don't really break this convention.
>

The purpose of including the toasted old data is only for the replica
identity,  but if you include it in the new tuple then it will affect the
general wal replay, basically, now you will have large detoasted data in
your new tuple which your are directly going to memcpy on the standby while
replaying so that will create corruption.   So basically, you can not
include this in the new tuple without changing a lot of logic around replay
which is completely useless.

So let this tuple be for a specific purpose and that is replica identity in
our case.


> And one small typo in the patch:
>
> The header above ExtractReplicaIdentity()
>
> Before:
>  * key_required should be false if caller knows that no replica identity
>  * columns changed value and it doesn't has any external data.
>  * It's always true in the DELETE case.
>
> After:
>  * key_required should be false if caller knows that no replica identity
>  * columns changed value and it doesn't have any external data.
>  * It's always true in the DELETE case.
>

Okay, I will change this.


-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com