Horton

Re: new heapcheck contrib module

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Mark Dilger <mark.dilger@enterprisedb.com>, Dilip Kumar <dilipbalaut@gmail.com>, Peter Geoghegan <pg@bowt.ie>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2020-07-31T19:05:23Z
Lists: pgsql-hackers 
Hi,

On 2020-07-31 12:42:51 -0400, Robert Haas wrote:
> On Fri, Jul 31, 2020 at 12:33 PM Andres Freund <andres@anarazel.de> wrote:
> > I'm not sure what I was thinking "back then", but right now I'd argue
> > that the best lock against vacuum isn't a SUE, but announcing the
> > correct ->xmin, so you can be sure that clog entries won't be yanked out
> > from under you. Potentially with the right flag sets to avoid old enough
> > tuples eing pruned.
> 
> Suppose we don't even do anything special in terms of advertising
> xmin. What can go wrong? To have a problem, we've got to be running
> concurrently with a vacuum that truncates clog. The clog truncation
> must happen before our XID lookups, but vacuum has to remove the XIDs
> from the heap before it can truncate. So we have to observe the XIDs
> before vacuum removes them, but then vacuum has to truncate before we
> look them up. But since we observe them and look them up while holding
> a ShareLock on the buffer, this seems impossible. What's the flaw in
> this argument?

The page could have been wrongly marked all-frozen. There could be
interactions between heap and toast table that are checked. Other bugs
could apply, like a broken hot chain or such.


> If we do need to do something special in terms of advertising xmin,
> how would you do it? Normally it happens by registering a snapshot,
> but here all we would have is an XID; specifically, the value of
> relfrozenxid that we observed.

An appropriate procarray or snapmgr function would probably suffice?

Greetings,

Andres Freund

Commits

  1. Add pg_amcheck, a CLI for contrib/amcheck.

  2. Refactor and generalize the ParallelSlot machinery.

  3. Generalize parallel slot result handling.

  4. Move some code from src/bin/scripts to src/fe_utils to permit reuse.

  5. Factor pattern-construction logic out of processSQLNamePattern.

  6. Doc: clean up verify_heapam() documentation.

  7. Fix more portability issues in new amcheck code.

  8. Fix portability issues in new amcheck test.

  9. Try to avoid a compiler warning about using fxid uninitialized.

  10. Extend amcheck to check heap pages.

  11. Adjust walsender usage of xlogreader, simplify APIs

  12. Improve checking of child pages in contrib/amcheck.

  13. Sanitize line pointers within contrib/amcheck.

  14. Fix possible sorting error when aborting use of abbreviated keys.