Creation of an empty table is not fsync'd at checkpoint
Heikki Linnakangas <hlinnaka@iki.fi>
From: Heikki Linnakangas <hlinnaka@iki.fi>
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2022-01-27T17:55:45Z
Lists: pgsql-hackers
Attachments
- 0001-Ensure-that-creation-of-an-empty-relfile-is-fsync-d-.patch (text/x-patch) patch 0001
If you create an empty table, it is not fsync'd. As soon as you insert a row to it, register_dirty_segment() gets called, and after that, the next checkpoint will fsync it. But before that, the creation itself is never fsync'd. That's obviously not great. The lack of an fsync is a bit hard to prove because it requires a hardware failure, or a simulation of it, and can be affected by filesystem options too. But I was able to demonstrate a problem with these steps: 1. Create a VM with two virtual disks. Use ext4, with 'data=writeback' option (I'm not sure if that's required). Install PostgreSQL on one of the virtual disks. 2. Start the server, and create a tablespace on the other disk: CREATE TABLESPACE foospc LOCATION '/data/heikki'; 3. Do this: CREATE TABLE foo (i int) TABLESPACE foospc; CHECKPOINT; 4. Immediately after that, kill the VM. I used: killall -9 qemu-system-x86_64 5. Restart the VM, restart PostgreSQL. Now when you try to use the table, you get an error: postgres=# select * from crashtest ; ERROR: could not open file "pg_tblspc/81921/PG_15_202201271/5/98304": No such file or directory I was not able to reproduce this without the tablespace on a different virtual disk, I presume because ext4 orders the writes so that the checkpoint implicitly always flushes the creation of the file to disk. I tried data=writeback but it didn't make a difference. But with a separate disk, it happens every time. I think the simplest fix is to call register_dirty_segment() from mdcreate(). As in the attached. Thoughts? - Heikki
Commits
-
Ensure that creation of an empty relfile is fsync'd at checkpoint.
- 6377f705cab2 11.21 landed
- 1b4f1c6f8a6c 12.16 landed
- acc8cdff4245 13.12 landed
- d85bf0719e71 14.9 landed
- e24c02e4d258 15.4 landed
- d431644b43fc 16.0 landed
- 4b4798e1384c 17.0 landed
-
Fix missing fsync of SLRU directories.
- aca74843e40f 14.0 cited