Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work
Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
To: Nathan Bossart <nathandbossart@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, "Bossart,
Nathan" <bossartn@amazon.com>, Tom Lane <tgl@sss.pgh.pa.us>, Julien Rouhaud <rjuju123@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-02-10T16:00:45Z
Lists: pgsql-hackers
Attachments
- v8-0001-Replace-ReadDir-with-ReadDirExtended.patch (application/octet-stream) patch v8-0001
On Fri, Feb 4, 2022 at 5:33 AM Nathan Bossart <nathandbossart@gmail.com> wrote: > > Thanks. I get it. For syncing map files, we don't want to tolerate any > > errors, whereas removal of the old map files (lesser than cutoff LSN) > > can be tolerated in CheckPointLogicalRewriteHeap. > > LGTM. Andres noted upthread [0] that the comment above sscanf() about > skipping editors' lock files might not be accurate. I don't think it's a > huge problem if sscanf() matches those files, but perhaps we can improve > the comment. > > [0] https://postgr.es/m/20220120194618.hmfd4kxkng2cgryh%40alap3.anarazel.de Andres comment from [0]: > An editor's lock file that starts with map- would presumably be the whole > filename plus an additional file-ending. But this check won't catch those. Agreed. sscanf checks can't detect the files named "whole filename plus an additional file-ending". I just checked with vi editor lock state file .0-14ED3B8.snap.swp [1], the log generated is [2]. I'm not sure exactly which editor would create a lockfile like "whole filename plus an additional file-ending". In any case, let's remove the editor's lock/state file from those comments and have just only "We just log a message if a file doesn't fit the pattern". Attached v8 patch with that change. [1] -rw------- 1 bharath bharath 12288 Feb 10 15:48 .0-14ED3B8.snap.swp -rw------- 1 bharath bharath 128 Feb 10 15:48 0-14ED518.snap -rw------- 1 bharath bharath 128 Feb 10 15:49 0-14ED518.snap.lockfile -rw------- 1 bharath bharath 128 Feb 10 15:49 0-14ED550.snap -rw------- 1 bharath bharath 128 Feb 10 15:49 0-14ED600.snap [2] 2022-02-10 15:48:47.938 UTC [1121678] LOG: could not parse file name "pg_logical/snapshots/.0-14ED3B8.snap.swp" Regards, Bharath Rupireddy.
Commits
-
Expand the use of get_dirent_type(), shaving a few calls to stat()/lstat()
- bfb9dfd93720 16.0 landed
-
fsync pg_logical/mappings in CheckPointLogicalRewriteHeap().
- fd48e5f5d3a1 13.6 landed
- f862cc09fad1 10.20 landed
- 2c15b29f7c22 11.15 landed
- 2b7dbe4bd587 14.2 landed
- 1c6d055ba77a 12.10 landed
- 1fabec7d7c38 15.0 landed
-
Introduce logical decoding.
- b89e151054a0 9.4.0 cited