Horton

Re: [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

Wood, Dan <hexpert@amazon.com>

View thread View on postgr.es
From: "Wood, Dan" <hexpert@amazon.com>
To: Michael Paquier <michael.paquier@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Peter Geoghegan <pg@bowt.ie>, "Wong, Yi Wen" <yiwong@amazon.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Robert Haas <robertmhaas@gmail.com>
Date: 2017-10-11T02:31:13Z
Lists: pgsql-hackers 
I found one glitch with our merge of the original dup row fix.  With that corrected AND Alvaro’s Friday fix things are solid.
No dup’s.  No index corruption.

Thanks so much. 

On 10/10/17, 7:25 PM, "Michael Paquier" <michael.paquier@gmail.com> wrote:

    On Tue, Oct 10, 2017 at 11:14 PM, Alvaro Herrera
    <alvherre@alvh.no-ip.org> wrote:
    > I was seeing just the reindex problem.  I don't see any more dups.
    >
    > But I've tried to reproduce it afresh now, and let it run for a long
    > time and nothing happened.  Maybe I made a mistake last week and
    > ran an unfixed version.  I don't see any more problems now.
    
    Okay, so that's one person more going to this trend, making three with
    Peter and I.
    
    >> If you are getting the dup rows consider the code in the block in
    >> heapam.c that starts with the comment “replace multi by update xid”.
    >>
    >> When I repro this I find that MultiXactIdGetUpdateXid() returns 0.
    >> There is an updater in the multixact array however the status is
    >> MultiXactStatusForNoKeyUpdate and not MultiXactStatusNoKeyUpdate.  I
    >> assume this is a preliminary status before the following row in the
    >> hot chain has it’s multixact set to NoKeyUpdate.
    >
    > Yes, the "For" version is the locker version rather than the actual
    > update.  That lock is acquired by EvalPlanQual locking the row just
    > before doing the update.  I think GetUpdateXid has no reason to return
    > such an Xid, since it's not an update.
    >
    >> Since a 0 is returned this does precede cutoff_xid and
    >> TransactionIdDidCommit(0) will return false.  This ends up aborting
    >> the multixact on the row even though the real xid is committed.  This
    >> sets XMAX to 0 and that row becomes visible as one of the dups.
    >> Interestingly the real xid of the updater is 122944 and the cutoff_xid
    >> is 122945.
    >
    > I haven't seen this effect. Please keep us updated if you're able to
    > verify corruption this way.
    
    Me neither. It would be nice to not live long with such a sword of Damocles.
    -- 
    Michael

Commits

  1. Backport addition of rs_old_rel to rewriteheap's state.

  2. Perform a lot more sanity checks when freezing tuples.

  3. Revert bogus fixes of HOT-freezing bug

  4. Fix traversal of half-frozen update chains

  5. Fix freezing of a dead HOT-updated tuple

  6. During index build, check and elog (not just Assert) for broken HOT chain.

  7. Fix WAL replay of locking an updated tuple

  8. Change the way we mark tuples as frozen.