Horton

pg_receivewal starting position

Ronan Dunklau <ronan.dunklau@aiven.io>

View thread View on postgr.es
From: Ronan Dunklau <ronan.dunklau@aiven.io>
To: pgsql-hackers@lists.postgresql.org
Date: 2021-07-27T05:50:39Z
Lists: pgsql-hackers 
Hello,

I've notived that pg_receivewal logic for deciding which LSN to start 
streaming at consists of:
  - looking up the latest WAL file in our destination folder, and resume from 
here
  - if there isn't, use the current flush location instead.

This behaviour surprised me when using it with a replication slot: I was 
expecting it to start streaming at the last flushed location from the 
replication slot instead. If you consider a backup tool which will take 
pg_receivewal's output and transfer it somewhere else, using the replication 
slot position would be the easiest way to ensure we don't miss WAL files.

Does that make sense ? 

I don't know if it should be the default, toggled by a command line flag, or if 
we even should let the user provide a LSN.

I'd be happy to implement any of that if we agree.

-- 
Ronan Dunklau

Commits

  1. Add TAP test for pg_receivewal with timeline switch

  2. Speed up TAP tests of pg_receivewal

  3. Allow pg_receivewal to stream from a slot's restart LSN

  4. Add replication command READ_REPLICATION_SLOT

  5. doc: Describe calculation method of streaming start for pg_receivewal

  6. Add PostgresNode::command_fails_like()