Horton

Thread

  1. verify checksums online

    Jeremy Schneider <schneider@ardentperf.com> — 2025-04-18T21:18:17Z

    i should know the answer to this... but asking anyway

i think there had been some mailing list discussions years ago? the
pg_checksum utility in core still can't do an online check right?

looking around, i think that michael's pg_checksums_ext still seems to
be available in debian repos, and it advertises online checks, which
makes me think that the core pg_checksum might still not have merged
any code for online checking yet

but i wanted to confirm with someone - my easiest options for online
verifying checksums would be to trigger pg_basebackup then check
pg_stat_database, or to install michael's utility?

-Jeremy

  2. Re: verify checksums online

    Ron <ronljohnsonjr@gmail.com> — 2025-04-18T21:32:19Z

    On Fri, Apr 18, 2025 at 5:18 PM Jeremy Schneider <schneider@ardentperf.com>
wrote:

> i should know the answer to this... but asking anyway
>
> i think there had been some mailing list discussions years ago? the
> pg_checksum utility in core still can't do an online check right?
>

https://www.postgresql.org/docs/17/app-pgchecksums.html

"The server must be shut down cleanly before running pg_checksums."


> looking around, i think that michael's pg_checksums_ext still seems to
> be available in debian repos, and it advertises online checks, which
> makes me think that the core pg_checksum might still not have merged
> any code for online checking yet
>
> but i wanted to confirm with someone - my easiest options for online
> verifying checksums would be to trigger pg_basebackup then check
> pg_stat_database, or to install michael's utility?
>

-- 
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

  3. Re: verify checksums online

    Jeremy Schneider <schneider@ardentperf.com> — 2025-04-18T22:18:26Z

    On Fri, 18 Apr 2025 17:32:19 -0400
Ron Johnson <ronljohnsonjr@gmail.com> wrote:

> On Fri, Apr 18, 2025 at 5:18 PM Jeremy Schneider
> <schneider@ardentperf.com> wrote:
> 
> > i think there had been some mailing list discussions years ago? the
> > pg_checksum utility in core still can't do an online check right?
> >
> 
> https://www.postgresql.org/docs/17/app-pgchecksums.html
> 
> "The server must be shut down cleanly before running pg_checksums."

Yes I read the docs. This utility has evolved over time including a
complete rename, and I was just doublechecking here that we didn't miss
a doc update in case this sentence applied to enable/disable but not to
verify. As I said I'm assuming it's all accurate but just doing the
paranoid double-check

-Jeremy

  4. Re: verify checksums online

    Greg Sabino Mullane <htamfids@gmail.com> — 2025-04-19T15:29:42Z

    On Fri, Apr 18, 2025 at 5:18 PM Jeremy Schneider <schneider@ardentperf.com>
wrote:

> but i wanted to confirm with someone - my easiest options for online
> verifying checksums would be to trigger pg_basebackup then check
> pg_stat_database, or to install michael's utility?
>

Your easiest option is to just use pgbackrest, which verifies checksums as
part of its process. So you are guaranteed to have every page verified at
least as often as every full backup.

Cheers,
Greg

--
Crunchy Data - https://www.crunchydata.com
Enterprise Postgres Software Products & Tech Support

  5. Re: verify checksums online

    Vijaykumar Jain <vijaykumarjain.github@gmail.com> — 2025-04-19T18:51:05Z

    iirc there used to be a Google snippet that did online checksum
verification. it worked pretty well , but I did not let it run the tests of
the extension, so it may not be 100% , I just used it for a quick check if
there were other problems visible.

if it's archived now well :)

https://github.com/google/pg_page_verification


Thanks,
Vijay

Open to work
Resume - Vijaykumar Jain <https://github.com/cabecada>

On Sat, Apr 19, 2025, 9:00 PM Greg Sabino Mullane <htamfids@gmail.com>
wrote:

> On Fri, Apr 18, 2025 at 5:18 PM Jeremy Schneider <schneider@ardentperf.com>
> wrote:
>
>> but i wanted to confirm with someone - my easiest options for online
>> verifying checksums would be to trigger pg_basebackup then check
>> pg_stat_database, or to install michael's utility?
>>
>
> Your easiest option is to just use pgbackrest, which verifies checksums as
> part of its process. So you are guaranteed to have every page verified at
> least as often as every full backup.
>
> Cheers,
> Greg
>
> --
> Crunchy Data - https://www.crunchydata.com
> Enterprise Postgres Software Products & Tech Support
>
>

  6. Re: verify checksums online

    Michael Banck <mbanck@gmx.net> — 2025-04-23T15:27:25Z

    Hi,

On Fri, Apr 18, 2025 at 03:18:26PM -0700, Jeremy Schneider wrote:
> On Fri, 18 Apr 2025 17:32:19 -0400
> Ron Johnson <ronljohnsonjr@gmail.com> wrote:
> 
> > On Fri, Apr 18, 2025 at 5:18 PM Jeremy Schneider
> > <schneider@ardentperf.com> wrote:
> > 
> > > i think there had been some mailing list discussions years ago? the
> > > pg_checksum utility in core still can't do an online check right?
> > 
> > https://www.postgresql.org/docs/17/app-pgchecksums.html
> > 
> > "The server must be shut down cleanly before running pg_checksums."
> 
> Yes I read the docs. This utility has evolved over time including a
> complete rename, and I was just doublechecking here that we didn't miss
> a doc update in case this sentence applied to enable/disable but not to
> verify. As I said I'm assuming it's all accurate but just doing the
> paranoid double-check

I proposed a patch some years ago but it was rejected or at least not
accepted without major changes[1]. There is an external version of
pg_checksums which can do online checksums verification here (using the
above patch): https://github.com/credativ/pg_checksums


Michael

[1] https://commitfest.postgresql.org/patch/1733/