Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>

From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
To: "Bossart, Nathan" <bossartn@amazon.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-01-13T06:02:53Z
Lists: pgsql-hackers
On Thu, Jan 13, 2022 at 3:47 AM Bossart, Nathan <bossartn@amazon.com> wrote:
>
> On 12/31/21, 4:44 AM, "Bharath Rupireddy" <bharath.rupireddyforpostgres@gmail.com> wrote:
> > Currently the server is erroring out when unable to remove/parse a
> > logical rewrite file in CheckPointLogicalRewriteHeap wasting the
> > amount of work the checkpoint has done and preventing the checkpoint
> > from finishing. This is unlike CheckPointSnapBuild does for snapshot
> > files i.e. it just emits a message at LOG level and continues if it is
> > unable to parse or remove the file. Attaching a small patch applying
> > the same idea to the mapping files.
>
> This seems reasonable to me.  AFAICT moving on to other files after an
> error shouldn't cause any problems.  In fact, it's probably beneficial
> to try to clean up as much as possible so that the files do not
> continue to build up.

Thanks for the review Nathan!

> The only feedback I have for the patch is that I don't think the new
> comments are necessary.

I borrowed the comments as-is from the CheckPointSnapBuild introduced
by the commit b89e15105. IMO, let the comments be there as they
explain why we are not emitting ERRORs, however I will leave it to the
committer to decide on that.

Regards,
Bharath Rupireddy.



Commits

  1. Expand the use of get_dirent_type(), shaving a few calls to stat()/lstat()

  2. fsync pg_logical/mappings in CheckPointLogicalRewriteHeap().

  3. Introduce logical decoding.