Re: [HACKERS] Restricting maximum keep segments by repslots

Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>

From: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
To: sk@zsrv.org
Cc: michael.paquier@gmail.com, andres@anarazel.de, peter.eisentraut@2ndquadrant.com, pgsql-hackers@postgresql.org
Date: 2018-01-11T06:59:10Z
Lists: pgsql-hackers

Attachments

Hello. Thank you for the comment.

(And sorry for the absense.)

At Fri, 22 Dec 2017 15:04:20 +0300, Sergei Kornilov <sk@zsrv.org> wrote in <337571513944260@web55j.yandex.ru>
> Hello
> I think limit wal in replication slots is useful in some cases. But first time i was confused with proposed terminology secured/insecured/broken/unknown state.

I'm not confident on the terminology. Suggestions are welcome on
the wording that makes more sense.

> patch -p1 gives some "Stripping trailing CRs from patch"
> messages for me, but applied to current HEAD and builds. After

Hmm. I wonder why I get that complaint so often. (It's rather
common? or caused by the MIME format of my mail?)  I'd say with
confidence that it is because you retrieved the patch file on
Windows mailer.

> little testing i understood the difference in
> secured/insecured/broken terminology. Secured means garantee to
> keep wal, insecure - wal may be deleted with next checkpoint,
> broken - wal already deleted.

Right. I'm sorry that I haven't written that clearly anywhere and
bothered you confirming that. I added documentation as the forth
patch.

> I think, we may split "secure" to "streaming"
> and... hmm... "waiting"? "keeping"? - according active flag for
> clarify and readable "status" field.

streaming / keeping and lost? (and unknown) Also "status" is
surely offers somewhat obscure meaning. wal_status (or
(wal_)availability) and min_keep_lsn maeke more sense?

The additional fields in pg_replication_slots have been changed
as the follows in the attached patch.

  confirmed_flush_lsn:
+ wal_status         : (streaming | keeping | lost | unknown)
+ min_keep_lsn       : <The oldest LSN that is available in WAL files>


The changes of documentation are seen in the following html files.

doc/src/sgml/html/warm-standby.html#STREAMING-REPLICATION-SLOTS
doc/src/sgml/html/runtime-config-replication.html#GUC-MAX-SLOT-WAL-KEEP-SIZE
doc/src/sgml/html/view-pg-replication-slots.html


One annoyance is that the min_keep_lsn always has the same value
among all slots.


regards,

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

Commits

  1. Save slot's restart_lsn when invalidated due to size

  2. Fix checkpoint signalling

  3. Check slot->restart_lsn validity in a few more places

  4. Allow users to limit storage reserved by replication slots

  5. Remove header noise from test_decoding test

  6. Rework WAL-reading supporting structs

  7. Flip argument order in XLogSegNoOffsetToRecPtr