Re: Online checksums verification in the backend

Magnus Hagander <magnus@hagander.net>

From: Magnus Hagander <magnus@hagander.net>
To: Andres Freund <andres@anarazel.de>
Cc: Robert Haas <robertmhaas@gmail.com>, Michael Paquier <michael@paquier.xyz>, Julien Rouhaud <rjuju123@gmail.com>, Justin Pryzby <pryzby@telsasoft.com>, Masahiko Sawada <masahiko.sawada@2ndquadrant.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Masahiko Sawada <sawada.mshk@gmail.com>
Date: 2020-11-03T08:31:20Z
Lists: pgsql-hackers
On Mon, Nov 2, 2020 at 8:35 PM Andres Freund <andres@anarazel.de> wrote:
>
> Hi,
>
> On 2020-11-02 12:35:30 -0500, Robert Haas wrote:
> > It feels really confusing to me that the user-exposed function here is
> > called pg_relation_check_pages(). How is the user supposed to
> > understand the difference between what this function does and what the
> > new verify_heapam() in amcheck does? The answer is that the latter
> > does far more extensive checks, but this isn't obvious from the SGML
> > documentation, which says only that the blocks are "verified," as if
> > an end-user can reasonably be expected to know what that means. It
> > seems likely to lead users to the belief that if this function passes,
> > they are in good shape, which is extremely far from being true. Just
> > look at what PageIsVerified() checks compared to what verify_heapam()
> > checks.
>
> Yea I had similar thoughts, it should just be called
> pg_checksum_verify_relation() or something.
>

+1.


> > In fact, I would argue that this functionality ought to live in
> > amcheck rather than core, though there could usefully be enabling
> > functions in core.
>
> I'm not really convinced by this though. It's not really AM
> specific - works for all types of relations with storage; don't really
> object either...

Yeah, I'm not sure about that one either. Also what would happen
if/when we get checksums on things that aren't even relations? (though
maybe that goes for other parts of amcheck at some point as well?)


-- 
 Magnus Hagander
 Me: https://www.hagander.net/
 Work: https://www.redpill-linpro.com/



Commits

  1. Revert pg_relation_check_pages()

  2. Fix incorrect placement of pfree() in pg_relation_check_pages()

  3. Add pg_relation_check_pages() to check on-disk pages of a relation

  4. Add CheckBuffer() to check on-disk pages without shared buffer loading

  5. Extend amcheck to check heap pages.