Horton

Re: new heapcheck contrib module

Mark Dilger <mark.dilger@enterprisedb.com>

View thread View on postgr.es
From: Mark Dilger <mark.dilger@enterprisedb.com>
To: Andrey Borodin <x4mmm@yandex-team.ru>, Peter Geoghegan <pg@bowt.ie>
Cc: Robert Haas <robertmhaas@gmail.com>, Stephen Frost <sfrost@snowman.net>, Michael Paquier <michael@paquier.xyz>, Amul Sul <sulamul@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2020-10-08T00:37:21Z
Lists: pgsql-hackers 

> On Oct 6, 2020, at 11:27 PM, Andrey Borodin <x4mmm@yandex-team.ru> wrote:
> 
> 
> 
>> 7 окт. 2020 г., в 04:20, Mark Dilger <mark.dilger@enterprisedb.com> написал(а):
>> 
>> 
>> 
>>> On Oct 5, 2020, at 5:24 PM, Mark Dilger <mark.dilger@enterprisedb.com> wrote:
>>> 
>>> - This version does not change clog handling, which leaves Andrey's concern unaddressed.  Peter also showed some support for (or perhaps just a lack of opposition to) doing more of what Andrey suggests.  I may come back to this issue, depending on time available and further feedback.
>> 
>> Attached is a patch set that includes the clog handling as discussed.  The 0001 and 0002 are effectively unchanged since version 16 posted yesterday, but this now includes 0003 which creates a non-throwing interface to clog, and 0004 which uses the non-throwing interface from within amcheck's heap checking functions.
>> 
>> I think this is a pretty good sketch for discussion, though I am unsatisfied with the lack of regression test coverage of verify_heapam in the presence of clog truncation.  I was hoping to have that as part of v17, but since it is taking a bit longer than I anticipated, I'll have to come back with that in a later patch.
>> 
> 
> Many thanks, Mark! I really appreciate this functionality. It could save me many hours of recreating clogs.

You are quite welcome, though the thanks may be premature.  I posted 0003 and 0004 patches mostly as concrete implementation examples that can be criticized.

> I'm not entire sure this message is correct: psprintf(_("xmax %u commit status is lost")
> It seems to me to be not commit status, but rather transaction status.

I have changed several such messages to say "transaction status" rather than "commit status".  I'll be posting it in a separate email, shortly.

Thanks for reviewing!

—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. Add pg_amcheck, a CLI for contrib/amcheck.

  2. Refactor and generalize the ParallelSlot machinery.

  3. Generalize parallel slot result handling.

  4. Move some code from src/bin/scripts to src/fe_utils to permit reuse.

  5. Factor pattern-construction logic out of processSQLNamePattern.

  6. Doc: clean up verify_heapam() documentation.

  7. Fix more portability issues in new amcheck code.

  8. Fix portability issues in new amcheck test.

  9. Try to avoid a compiler warning about using fxid uninitialized.

  10. Extend amcheck to check heap pages.

  11. Adjust walsender usage of xlogreader, simplify APIs

  12. Improve checking of child pages in contrib/amcheck.

  13. Sanitize line pointers within contrib/amcheck.

  14. Fix possible sorting error when aborting use of abbreviated keys.