Horton

Re: prevent immature WAL streaming

Amul Sul <sulamul@gmail.com>

View thread View on postgr.es
From: Amul Sul <sulamul@gmail.com>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: "Bossart, Nathan" <bossartn@amazon.com>, "Jakub.Wartak@tomtom.com" <Jakub.Wartak@tomtom.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, Ryo Matsumura <matsumura.ryo@fujitsu.com>, "andres@anarazel.de" <andres@anarazel.de>, "masao.fujii@oss.nttdata.com" <masao.fujii@oss.nttdata.com>, "mengjuan.cmj@alibaba-inc.com" <mengjuan.cmj@alibaba-inc.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>
Date: 2021-10-13T06:00:18Z
Lists: pgsql-hackers

Attachments

Hi,

On Thu, Oct 7, 2021 at 6:57 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2021-Oct-07, Amul Sul wrote:
>
> > Make sense, thanks for the explanation.
>
> You're welcome.  Also, I forgot: thank you for taking the time to review
> the code.  Much appreciated.

:)

>
>

I have one more question, regarding the need for other global
variables i.e. abortedRecPtr.  (Sorry for coming back after so long.)

Instead of abortedRecPtr point, isn't enough to write
overwrite-contrecord at XLogCtl->lastReplayedEndRecPtr?  I think both
are pointing to the same location then can't we use
lastReplayedEndRecPtr instead of abortedRecPtr to write
overwrite-contrecord and remove need of extra global variable, like
attached?

You might wonder why I am so concerned about the global variable. The
reason is that I am working on another thread[1] where we are trying
to club all the WAL write operations that happen at the end of
StartupXLOG into a separate function. In the future, we might want to
allow executing this function from other processes (e.g.
Checkpointer). For that, we need to remove the dependency of those WAL
write operations having on the global variables which are mostly valid
in the startup process. The easiest way to do that is simply copy all
the global variables into shared memory but that will not be an
optimised solution, the goal is to try to see if we could leverage the existing
information available into shared memory. I would be grateful if you
could share your thoughts on the same, thanks.

Regards,
Amul

1] https://postgr.es/m/CAAJ_b97KZzdJsffwRK7w0XU5HnXkcgKgTR69t8cOZztsyXjkQw@mail.gmail.com

Commits

  1. Fix determination of broken LSN in OVERWRITTEN_CONTRECORD

  2. Fix instability in 026_overwrite_contrecord.pl test.

  3. Fix typo

  4. Change recently added test code for stability

  5. Remove unstable, unnecessary test; fix typo

  6. Repair two portability oversights of new test

  7. Fix WAL replay in presence of an incomplete record

  8. Document XLOG_INCLUDE_XID a little better

  9. Revert "Avoid creating archive status ".ready" files too early"

  10. Avoid creating archive status ".ready" files too early

  11. Fix scenario where streaming standby gets stuck at a continuation record.

  12. Teach xlogreader to follow timeline switches

  13. Forbid using pg_xlogfile_name() and pg_xlogfile_name_offset() during