Re: .ready and .done files considered harmful

Dipesh Pandit <dipesh.pandit@gmail.com>

From: Dipesh Pandit <dipesh.pandit@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: "Bossart, Nathan" <bossartn@amazon.com>, Jeevan Ladhe <jeevan.ladhe@enterprisedb.com>, Stephen Frost <sfrost@snowman.net>, Andres Freund <andres@anarazel.de>, Hannu Krosing <hannuk@google.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2021-07-28T10:48:26Z
Lists: pgsql-hackers
Hi,

> I don't think it's great that we're using up SIGINT for this purpose.
> There aren't that many signals available at the O/S level that we can
> use for our purposes, and we generally try to multiplex them at the
> application layer, e.g. by setting a latch or a flag in shared memory,
> rather than using a separate signal. Can we do something of that sort
> here? Or maybe we don't even need a signal. ThisTimeLineID is already
> visible in shared memory, so why not just have the archiver just check
> and see whether it's changed, say via a new accessor function
> GetCurrentTimeLineID()?

As of now shared memory is not attached to the archiver. Archiver cannot
access ThisTimeLineID or a flag available in shared memory.

    if (strcmp(argv[1], "--forkbackend") == 0 ||

        strcmp(argv[1], "--forkavlauncher") == 0 ||

        strcmp(argv[1], "--forkavworker") == 0 ||

        strcmp(argv[1], "--forkboot") == 0 ||

        strncmp(argv[1], "--forkbgworker=", 15) == 0)

        PGSharedMemoryReAttach();

    else

        PGSharedMemoryNoReAttach();

This is the reason we have thought of sending a notification to the
archiver if
there is a timeline switch. Should we consider attaching shared memory to
archiver process or explore more on notification mechanism to avoid
using SIGINT?

Thanks,
Dipesh

Commits

  1. Reduce overhead of renaming archive status files.

  2. Improve performance of pgarch_readyXlog() with many status files.

  3. Prioritize history files when archiving