Re: Logical replication 'invalid memory alloc request size 1585837200' after upgrading to 17.5

Masahiko Sawada <sawada.mshk@gmail.com>

From: Masahiko Sawada <sawada.mshk@gmail.com>
To: vignesh C <vignesh21@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, Duncan Sands <duncan.sands@deepbluecap.com>, "pgsql-bugs@lists.postgresql.org" <pgsql-bugs@lists.postgresql.org>
Date: 2025-06-05T21:59:12Z
Lists: pgsql-bugs

Attachments

On Thu, Jun 5, 2025 at 3:28 AM vignesh C <vignesh21@gmail.com> wrote:
>
> On Thu, 5 Jun 2025 at 03:19, Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> > Thank you for updating the patch. I have some comments and questions:
> >
> > In ReorderBufferAbort():
> >
> >         /*
> >          * We might have decoded changes for this transaction that could load
> >          * the cache as per the current transaction's view (consider DDL's
> >          * happened in this transaction). We don't want the decoding of future
> >          * transactions to use those cache entries so execute invalidations.
> >          */
> >         if (txn->ninvalidations > 0)
> >             ReorderBufferImmediateInvalidation(rb, txn->ninvalidations,
> >                                                txn->invalidations);
> >
> > I think that if the txn->invalidations_distributed is overflowed, we
> > would miss executing the txn->invalidations here. Probably the same is
> > true for ReorderBufferForget() and ReorderBufferInvalidate().
>
> I'm accumulating the invalidations in txn->invalidations irrespective
> of RBTXN_INVAL_OVERFLOWED txn.

Agreed with this change.

>
> > ---
> > I'd like to make it clear again which case we need to execute
> > txn->invalidations as well as txn->invalidations_distributed (like in
> > ReorderBufferProcessTXN()) and which case we need to execute only
> > txn->invalidations (like in ReorderBufferForget() and
> > ReorderBufferAbort()). I think it might be worth putting some comments
> > about overall strategy somewhere.
>
> I have added  comments for this, feel free to reword it if some
> changes are required.
>
> The attached v11 version patch has the changes for the same.

I think the patch is getting into good shape. I've attached a patch
that includes changes I recommend. For example, it's better to rename
RBTXN_INVAL_OVERFLOWED to RBTXN_DISTR_INVAL_OVERFLOWED, and it
includes some comment updates. Please review them.

Regards,

-- 
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Fix cache-dependent test failures in logical decoding.

  2. Fix re-distributing previously distributed invalidation messages during logical decoding.

  3. Fix data loss in logical replication.