Re: BUG #17064: Parallel VACUUM operations cause the error "global/pg_filenode.map contains incorrect checksum"
Thomas Munro <thomas.munro@gmail.com>
From: Thomas Munro <thomas.munro@gmail.com>
To: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Alexander Lakhin <exclusion@gmail.com>, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Date: 2021-06-28T02:45:56Z
Lists: pgsql-bugs
Attachments
- 0001-Remove-control-file-dependency-on-512-byte-sectors.patch (text/x-patch) patch 0001
> We haven't heard of broken control files from the field, so that doesn't > seem to be a problem in practice, at least not yet. Still, I would sleep > better if the control file had more redundancy. For example, have two > copies of it on disk. At startup, read both copies, and if they're both > valid, ignore the one with older timestamp. When updating it, write over > the older copy. That way, if you crash in the middle of updating it, the > old copy is still intact. Seems like a good idea. I somehow doubt that accessing pmem through old school read()/write() interfaces is the future of databases, but ideally this should work correctly, and the dependency is indeed unnecessary if we are prepared to jump through more hoops in just a couple of places. There may also be other benefits. In hindsight, it's a bit strange that we don't have explicit documentation of this requirement. There is some related (and rather dated) discussion of sectors in wal.sgml but nothing to say that we need 512 byte atomic sectors for correct operation, unless I've managed to miss it (even though it's well known among people who read the source code). I experimented with a slightly different approach, attached, and a TAP test to exercise it. Instead of alternating between two copies, I tried writing out both copies every time with a synchronisation barrier in between (the same double-write principle some other database uses to deal with torn data pages). I think it's mostly equivalent to your scheme, though the updates are of course slower. I was thinking that there may be other benefits to having two copies of the "current" version around, for resilience (though perhaps they should be in separate files, not done here), and maybe it's better to avoid having to invent a timestamp scheme. Or maybe the two ideas should be combined: when both CRC checks pass, you could still be more careful which one you choose than I have been here. Or maybe trying to be resilient against handwavy unknown forms of corruption is a waste of time. I'm not proposing anything here, I was just trying out ideas, for discussion.
Commits
-
Another fix to relmapper race condition.
- 5956795cb5be 9.6.23 landed
- fbbc48995d05 10.18 landed
- eb3bd243a245 11.13 landed
- 8b01a403c5cc 12.8 landed
- 6fb377e5f74b 13.4 landed
- 9b8ed0f52bff 14.0 landed
-
Prevent race condition while reading relmapper file.
- 85ae8ccb2fb3 9.6.23 landed
- a5eacd86b715 10.18 landed
- c78bb32c19fe 11.13 landed
- caac1965077f 12.8 landed
- 2a0ab13f8dd1 13.4 landed
- b6d8d2073f22 14.0 landed