Horton

Re: Restricting maximum keep segments by repslots

Petr Jelinek <petr.jelinek@2ndquadrant.com>

View thread View on postgr.es
From: Petr Jelinek <petr.jelinek@2ndquadrant.com>
To: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>, pgsql-hackers@postgresql.org
Date: 2017-02-28T04:27:05Z
Lists: pgsql-hackers 
On 28/02/17 04:27, Kyotaro HORIGUCHI wrote:
> Hello.
> 
> Although replication slot is helpful to avoid unwanted WAL
> deletion, on the other hand it can cause a disastrous situation
> by keeping WAL segments without a limit. Removing the causal
> repslot will save this situation but it is not doable if the
> standby is active. We should do a rather complex and forcible
> steps to relieve the situation especially in an automatic
> manner. (As for me, specifically in an HA cluster.)
> 

I agree that that it should be possible to limit how much WAL slot keeps.

> This patch adds a GUC to put a limit to the number of segments
> that replication slots can keep. Hitting the limit during
> checkpoint shows a warining and the segments older than the limit
> are removed.
> 
>> WARNING:  restart LSN of replication slots is ignored by checkpoint
>> DETAIL:  Some replication slots lose required WAL segnents to continue.
> 

However this is dangerous as logical replication slot does not consider
it error when too old LSN is requested so we'd continue replication,
hiding data loss.

-- 
  Petr Jelinek                  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services

Commits

  1. Save slot's restart_lsn when invalidated due to size

  2. Fix checkpoint signalling

  3. Check slot->restart_lsn validity in a few more places

  4. Allow users to limit storage reserved by replication slots

  5. Remove header noise from test_decoding test

  6. Rework WAL-reading supporting structs

  7. Flip argument order in XLogSegNoOffsetToRecPtr