Re: Online checksums verification in the backend

Robert Haas <robertmhaas@gmail.com>

From: Robert Haas <robertmhaas@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Michael Paquier <michael@paquier.xyz>, Julien Rouhaud <rjuju123@gmail.com>, Justin Pryzby <pryzby@telsasoft.com>, Masahiko Sawada <masahiko.sawada@2ndquadrant.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Masahiko Sawada <sawada.mshk@gmail.com>
Date: 2020-11-03T16:52:01Z
Lists: pgsql-hackers
On Mon, Nov 2, 2020 at 2:35 PM Andres Freund <andres@anarazel.de> wrote:
> Wouldn't this be better served by having a ReadBufferExtended() flag,
> preventing erroring out and zeroing the buffer? I'm not sure that
> handling both the case where the buffer contents need to be valid and
> the one where it doesn't will make for a good API.

I'm not sure. The goal I had in mind was giving a caller a way to get
a copy of a buffer even if it's one we wouldn't normally admit into
shared_buffers. I think it's probably a bad idea to allow for a back
door where things that fail PageIsVerified() can nevertheless escape
into the buffer, but that doesn't mean a checker or recovery tool
shouldn't be allowed to see them.

> > If the buffer is in shared buffers, we could take a share-lock
> > on the buffer and copy the contents of the page as it exists on disk,
> > and then still check it.
>
> Don't think we need a share lock. That still allows the buffer to be
> written out (and thus a torn read). What we need is to set
> BM_IO_IN_PROGRESS on the buffer in question - only one backend can set
> that. And then unset that again, without unsetting
> BM_DIRTY/BM_JUST_DIRTIED.

OK.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Commits

  1. Revert pg_relation_check_pages()

  2. Fix incorrect placement of pfree() in pg_relation_check_pages()

  3. Add pg_relation_check_pages() to check on-disk pages of a relation

  4. Add CheckBuffer() to check on-disk pages without shared buffer loading

  5. Extend amcheck to check heap pages.