Re: [HACKERS] Restricting maximum keep segments by repslots
Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
From: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
To: peter.eisentraut@2ndquadrant.com
Cc: sawada.mshk@gmail.com, pgsql-hackers@lists.postgresql.org,
thomas.munro@enterprisedb.com, sk@zsrv.org, michael.paquier@gmail.com,
andres@anarazel.de
Date: 2018-09-13T09:29:31Z
Lists: pgsql-hackers
Attachments
- v9-0001-Add-WAL-relief-vent-for-replication-slots.patch (text/x-patch)
Hello. Thank you for the comments, Sawada-san, Peter. At Mon, 10 Sep 2018 19:52:24 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp> wrote in <20180910.195224.22629595.horiguchi.kyotaro@lab.ntt.co.jp> > At Thu, 6 Sep 2018 22:32:21 +0200, Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote in <29bbd79d-696b-509e-578a-0fc38a3b9405@2ndquadrant.com> > Thanks for pointing that. That's a major cause of confusion. Does > the following make sense? > > > Specify the maximum size of WAL files that <link > > linkend="streaming-replication-slots">replication slots</link> > > are allowed to retain in the <filename>pg_wal</filename> > > directory at checkpoint time. If > > <varname>max_slot_wal_keep_size</varname> is zero (the > > default), replication slots retain unlimited size of WAL files. > + If restart_lsn of a replication slot gets behind more than that > + bytes from the current LSN, the standby using the slot may not > + be able to reconnect due to removal of required WAL records. ... > > Also, I don't think 0 is a good value for the default behavior. 0 would > > mean that a slot is not allowed to retain any more WAL than already > > exists anyway. Maybe we don't want to support that directly, but it's a > > valid configuration. So maybe use -1 for infinity. > > In realtion to the reply just sent to Sawada-san, remain of a > slot can be at most 16MB in the 0 case with the default segment > size. So you're right in this sense. Will fix in the coming > version. Thanks. I did the following thinkgs in the new version. - Changed the disable (or infinite) and default value of max_slot_wal_keep_size to -1 from 0. (patch 1, 2. guc.c, xlog.c: GetOldestKeepSegment()) - Fixed documentation for max_slot_wal_keep_size tomention what happnes when WAL exceeds the size, and additional rewrites. (patch 4, catalogs.sgml, config.sgml) - Folded parameter list of GetOldestKeepSegment(). (patch 1, 2. xlog.c) - Provided the plural form of errdetail of checkpoint-time warning. (patch 1, xlog.c: KeepLogSeg()) - Some cosmetic change and small refactor. (patch 1, 2, 3) regards. -- Kyotaro Horiguchi NTT Open Source Software Center
Commits
-
Save slot's restart_lsn when invalidated due to size
- 12e52ba5a76e 13.0 landed
- 0188bb82531f 14.0 landed
-
Fix checkpoint signalling
- 1816a1c6ffe4 13.0 landed
-
Check slot->restart_lsn validity in a few more places
- d0abe78d8427 13.0 landed
-
Allow users to limit storage reserved by replication slots
- c6550776394e 13.0 landed
-
Remove header noise from test_decoding test
- 69360b34589b 13.0 landed
-
Rework WAL-reading supporting structs
- 709d003fbd98 13.0 cited
-
Flip argument order in XLogSegNoOffsetToRecPtr
- a22445ff0be2 12.0 cited