Horton
Menu

Re: Changing the state of data checksums in a running cluster

Daniel Gustafsson <daniel@yesql.se>

View thread
From: Daniel Gustafsson <daniel@yesql.se>
To: Tomas Vondra <tomas@vondra.me>
Cc: Andres Freund <andres@anarazel.de>, Bernd Helmle <mailings@oopsware.de>, Michael Paquier <michael@paquier.xyz>, Michael Banck <mbanck@gmx.net>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2026-03-11T23:56:07Z
Lists: pgsql-hackers

Attachments

> On 6 Feb 2026, at 18:15, Tomas Vondra <tomas@vondra.me> wrote:

> I spent a bit more time looking at this today, and I figured out a
> simpler way to "cause trouble" by using PITR. I don't know if this has
> the same root cause as the failures in the 006 TAP test, but I find it
> interesting and I think it highlights some issues with the new patch.

I've encoded some of this into a new test suite in the attached patchset,
having it in a repeatable test should make it easier to reproduce and reason
about.

Attached is a rebase of the patchset which further refines the switch to using
XLOG_CHECKPOINT_* records which was proposed by Andres upstream.  The errors I
encountered could be traced to replay across multiple state transitions (with
at least one case of ON->OFF), and re-introducing checkpoints at on and off
state transitions helped removing those.

* A lot of code has been moved out from xlog.c and into the file which contains
  the code for the datachecksumsworker.  This file has in turn been renamed to
  signal what it's used for.  It should probably be moved from backend/postmaster
  but it's not clear to me where it should go.
* Improved signalhandling in the worker code.
* A new testsuite for PITR is added, which is based on Tomas' findings.  This
  can be further increased and refined but it's at least a start.
* Another new testsuite which changes full_page_writes settings is also added.
  This one should perhaps be rolled into 001_basic, not sure.
* Tests 006 and 007, the pgbench "torture test" suites, are now being run in a
  pared-down fashion when PG_TEST_EXTRA isn't defined, rather than being skipped
  entirely
* The minimum max_wal_size in tests is now 32 instead of 64.
* Lots of polish and small tweaking and new comments

There are a few TODOs, the more interesting ones include:

* The change to XLOG_CHECKPOINT_REDO to move the wal_level into a proper record
  structure should be pulled out as a 0001 patch as it's an cleanup that has
  value on its own.
* Try to move even more code awy from xlog.c
* Address, and remove, all XXX comments (most of which stem from a much earlier
  version of the patchset).
* The code for emitting, and waiting on, the procsignalbarrier exists in
  multiple copies which needs to be refactored into a single one.
* Tests failed in Autoconf on CI, which I need to fix.

My next step will be addressing these TODOs, but I wanted to get a new version
on the list sooner rather than later.

--
Daniel Gustafsson

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Use correct datatype for PID

  2. Improve comments in online checksums code

  3. Fix checksum state transition during promotion

  4. Fix regex searching for page verification failures in tests

  5. Apply data-checksum worker throttling parameters

  6. Skip WAL for unlogged main fork during online checksum enable

  7. Fix data_checksum GUC show_hook

  8. Improve database detection logic in datachecksumsworker

  9. Improve handling of concurrent checksum requests

  10. Typo and spelling fixups for online checksums

  11. Fix invalid checksum state transition in checkpoints

  12. Handle data_checksum state changes during launcher_exit

  13. Test improvements for online checksums

  14. Prevent pg_enable/disable_data_checksums() on standby

  15. Test stabilization for online checksums

  16. Make data checksum tests more resilient for slow machines

  17. Formalize WAL record for XLOG_CHECKPOINT_REDO

  18. Revert "Get rid of WALBufMappingLock"

  19. Get rid of WALBufMappingLock

  20. Improve grammar of options for command arrays in TAP tests