Re: [PATCH 13/16] Introduction of pair of logical walreceiver/sender

On Friday, June 29, 2012 05:16:11 PM Heikki Linnakangas wrote:
> On 13.06.2012 14:28, Andres Freund wrote:
> > A logical WALReceiver is started directly by Postmaster when we enter
> > PM_RUN state and the new parameter multimaster_conninfo is set. For now
> > only one of those is started, but the code doesn't rely on that. In
> > future multiple ones should be allowed.
> 
> Could the receiver-side of this be handled as an extra daemon:
> http://archives.postgresql.org/message-id/CADyhKSW2uyrO3zx-tohzRhN5-vaBEfKN
> HyvLG1yp7=cx_YH9UA@mail.gmail.com
Well, I think it depends on what the protocol turns out to be. In the 
prototype we used the infrastructure from walreceiver which reduced the 
required code considerably.

> In general, I feel that the receiver-side could live outside core.
I think it should be possible to write receivers outside core, but one 
sensible implementation should be in-core.

> The sender-side needs to be at least somewhat integrated into the walsender
> stuff, and there are changes to the WAL records etc. that are hard to do
> outside, but AFAICS the stuff to receive changes is pretty high-level
> stuff. 
> None of that needs to be in implemented inside a PostgreSQL server.
If you want robust and low-overhead crash recovery you need (at least I think 
so) tighter integration into postgres. To be sure that you pick of where you 
stopped after a crash you need to have a state synchronized to the commits 
into the receiving side. So you either always write to another table and 
analyze that afterwards - which imo sucks - or you integrate it with the 
commit record. Which needs integration into pg.

Greetings,

Andres
-- 
 Andres Freund	                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services