Re: Adding CommandID to heap xlog records
Heikki Linnakangas <hlinnaka@iki.fi>
From: Heikki Linnakangas <hlinnaka@iki.fi>
To: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>,
Bruce Momjian <bruce@momjian.us>, Tom Lane <tgl@sss.pgh.pa.us>,
Andres Freund <andres@anarazel.de>, vignesh C <vignesh21@gmail.com>,
Ian Lawrence Barwick <barwick@gmail.com>
Date: 2023-02-28T13:52:26Z
Lists: pgsql-hackers
Attachments
- 0001-Improve-comment-on-why-we-need-ctid-cmin-cmax-mappin.patch (text/x-patch) patch 0001
- 0002-Remove-redundant-check-for-fast_forward.patch (text/x-patch) patch 0002
- 0003-Remove-combocid-field-in-logical-decode-that-was-jus.patch (text/x-patch) patch 0003
- 0004-Include-command-ID-in-heapam-records-on-catalog-tabl.patch (text/x-patch) patch 0004
I took another stab at this from a different angle, and tried to use this to simplify logical decoding. The theory was that if we included the command ID in the WAL records, we wouldn't need the separate HEAP2_NEW_CID record anymore, and could remove much of the code in reorderbuffer.c that's concerned with tracking ctid->(cmin,cmax) mapping. Unfortunately, it didn't work out. Here's one problem: Insert with cmin 1 Commit Delete the same tuple with cmax 2. Abort Even if we store the cmin in the INSERT record, and set it on the tuple on replay, the DELETE overwrites it. That's OK for the original transactions, because they only look at the cmin/cmax of their own transaction, but it's a problem for logical decoding. If we see the inserted tuple during logical decoding, we need the cmin of the tuple. We could still just replace the HEAP2_NEW_CID records with the CIDs in the heap INSERT/UPDATE/DELETE records, and use that information to maintain the ctid->(cmin,cmax) mapping in reorderbuffer.c like we do today. But that doesn't really simplify reorderbuffer.c much. Attached is a patch for that, for the archives sake. Another problem with that is that logical decoding needs slightly different information than what we store on the tuples on disk. My original motivation for this was for Neon, which needs the WAL replay to restore the same CID as what's stored on disk, whether it's cmin, cmax or combocid. But for logical decoding, we need the cmin or cmax, *not* the combocid. To cater for both uses, we'd need to include both the original cmin/cmax and the possible combocid, which again makes it more complicated. So unfortunately I don't see much opportunity to simplify logical decoding with this. However, please take a look at the first two patches attached. They're tiny cleanups that make sense on their own. - Heikki
Commits
-
Remove redundant check for fast_forward.
- e251e780bfe6 17.0 landed
-
Improve comment on why we need ctid->(cmin,cmax) mapping.
- a0dd4c95b9df 17.0 landed