Re: Skip collecting decoded changes of already-aborted transactions

On Wed, Dec 11, 2024 at 10:01 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Thu, Dec 12, 2024 at 11:08 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Wed, Dec 11, 2024 at 8:21 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> > >
> > > On Wed, Dec 11, 2024 at 3:18 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > > >
> > > > On Mon, Dec 9, 2024 at 10:19 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> > >
> > > > > >
> > > > > > If the largest transaction is non-streamable, won't the transaction
> > > > > > returned by ReorderBufferLargestTXN() in the other case already
> > > > > > suffice the need?
> > > > >
> > > > > I see your point, but I don’t think it’s quite the same. When
> > > > > ReorderBufferCanStartStreaming() is true, the function
> > > > > ReorderBufferLargestStreamableTopTXN() looks for the largest
> > > > > transaction among those that have a base_snapshot. So, if the largest
> > > > > transaction is aborted but hasn’t yet received a base_snapshot, it
> > > > > will instead select the largest transaction that does have a
> > > > > base_snapshot, which could be significantly smaller than the largest
> > > > > aborted transaction.
> > > >
> > > > IIUC the transaction entries in reorderbuffer have the base snapshot
> > > > before decoding the first change (see SnapBuildProcessChange()). In
> > > > which case the transaction doesn't have the base snapshot and has the
> > > > largest amount of changes? Subtransaction entries could transfer its
> > > > base snapshot to its parent transaction entry but such subtransactions
> > > > will be picked by ReorderBufferLargestTXN().
> > > >
> > > IIRC, there could be cases where reorder buffers of transactions can
> > > grow in size without having a base snapshot, I think transactions
> > > doing DDLs and generating a lot of INVALIDATION messages could fall in
> > > such a category.
> > >
> >
> > Are we recording such changes in the reorder buffer? If so, can you
> > please share how?
>
> xact_decode, do add the XLOG_XACT_INVALIDATIONS in the reorder buffer
> and for such changes we don't call SnapBuildProcessChange() that means
> it is possible to collect such changes in reorder buffer without
> setting the base_snapshot

DDLs write not only XLOG_XACT_INVALIDATIONS but also system catalog
changes. I think that when decoding these system catalog changes, we
end up calling SnapBuildProcessChange(). I understand that decoding
XLOG_XACT_INVALIDATIONS doesn't call SnapBuildProcessChange() but
queues invalidation messages to the reorderbuffer, but I still don't
understand cases where a transaction entry is quite big and has only a
lot of invalidation messages.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com