Thread

Replication logging

Magnus Hagander <magnus@hagander.net> — 2011-01-16T14:19:26Z

Currently, replication connections *always* logs something like:
LOG:  replication connection authorized: user=mha host=[local]

There's no way to turn that off.

I can't find the reasoning behind this - why is this one not
controlled by log_connections like normal ones? There's a comment in
the code that says this is intentional, but I can't figure out why...

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Replication logging

Robert Haas <robertmhaas@gmail.com> — 2011-01-17T02:06:49Z

On Sun, Jan 16, 2011 at 9:19 AM, Magnus Hagander <magnus@hagander.net> wrote:
> Currently, replication connections *always* logs something like:
> LOG:  replication connection authorized: user=mha host=[local]
>
> There's no way to turn that off.
>
> I can't find the reasoning behind this - why is this one not
> controlled by log_connections like normal ones? There's a comment in
> the code that says this is intentional, but I can't figure out why...

Because it's reasonably likely that you'd want to log replication
connections but not regular ones?  On the theory that replication is
more important than an ordinary login?

What do you have in mind?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: Replication logging

Magnus Hagander <magnus@hagander.net> — 2011-01-17T06:53:15Z

On Mon, Jan 17, 2011 at 03:06, Robert Haas <robertmhaas@gmail.com> wrote:
> On Sun, Jan 16, 2011 at 9:19 AM, Magnus Hagander <magnus@hagander.net> wrote:
>> Currently, replication connections *always* logs something like:
>> LOG:  replication connection authorized: user=mha host=[local]
>>
>> There's no way to turn that off.
>>
>> I can't find the reasoning behind this - why is this one not
>> controlled by log_connections like normal ones? There's a comment in
>> the code that says this is intentional, but I can't figure out why...
>
> Because it's reasonably likely that you'd want to log replication
> connections but not regular ones?  On the theory that replication is
> more important than an ordinary login?

Well, a superuser connection is even worse, but we don't hard-code
logging of those.

> What do you have in mind?

Either having it controlled by log_connections, or perhaps have a
log_highpriv_connections that controls replication *and* superuser, to
be somewhat consistent.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Replication logging

Robert Haas <robertmhaas@gmail.com> — 2011-01-17T13:00:51Z

On Mon, Jan 17, 2011 at 1:53 AM, Magnus Hagander <magnus@hagander.net> wrote:
> On Mon, Jan 17, 2011 at 03:06, Robert Haas <robertmhaas@gmail.com> wrote:
>> On Sun, Jan 16, 2011 at 9:19 AM, Magnus Hagander <magnus@hagander.net> wrote:
>>> Currently, replication connections *always* logs something like:
>>> LOG:  replication connection authorized: user=mha host=[local]
>>>
>>> There's no way to turn that off.
>>>
>>> I can't find the reasoning behind this - why is this one not
>>> controlled by log_connections like normal ones? There's a comment in
>>> the code that says this is intentional, but I can't figure out why...
>>
>> Because it's reasonably likely that you'd want to log replication
>> connections but not regular ones?  On the theory that replication is
>> more important than an ordinary login?
>
> Well, a superuser connection is even worse, but we don't hard-code
> logging of those.

From a security perspective, perhaps; but not from an "oh crap my
replication slave can't connect I'm hosed if the server crashes"
perspective.

>> What do you have in mind?
>
> Either having it controlled by log_connections, or perhaps have a
> log_highpriv_connections that controls replication *and* superuser, to
> be somewhat consistent.

-1.  We could provide an option to turn this on and off, but I
wouldn't want it merged with log_connections or logging of superuser
connections.

Incidentally, I think ClientAuthentication_hook is sufficiently
powerful to allow logging of superuser connections but no others, if
someone wanted to write a contrib module.  That doesn't necessarily
mean an in-core facility wouldn't be useful too, but it's at least
worth thinking about using the hook.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: Replication logging

Magnus Hagander <magnus@hagander.net> — 2011-01-17T13:58:22Z

On Mon, Jan 17, 2011 at 14:00, Robert Haas <robertmhaas@gmail.com> wrote:
> On Mon, Jan 17, 2011 at 1:53 AM, Magnus Hagander <magnus@hagander.net> wrote:
>> On Mon, Jan 17, 2011 at 03:06, Robert Haas <robertmhaas@gmail.com> wrote:
>>> On Sun, Jan 16, 2011 at 9:19 AM, Magnus Hagander <magnus@hagander.net> wrote:
>>>> Currently, replication connections *always* logs something like:
>>>> LOG:  replication connection authorized: user=mha host=[local]
>>>>
>>>> There's no way to turn that off.
>>>>
>>>> I can't find the reasoning behind this - why is this one not
>>>> controlled by log_connections like normal ones? There's a comment in
>>>> the code that says this is intentional, but I can't figure out why...
>>>
>>> Because it's reasonably likely that you'd want to log replication
>>> connections but not regular ones?  On the theory that replication is
>>> more important than an ordinary login?
>>
>> Well, a superuser connection is even worse, but we don't hard-code
>> logging of those.
>
> From a security perspective, perhaps; but not from an "oh crap my
> replication slave can't connect I'm hosed if the server crashes"
> perspective.

True, there are more than one ways to look at them.

That doesn't mean one is more important than the other though, so they
should be equally configurable, imho.


>>> What do you have in mind?
>>
>> Either having it controlled by log_connections, or perhaps have a
>> log_highpriv_connections that controls replication *and* superuser, to
>> be somewhat consistent.
>
> -1.  We could provide an option to turn this on and off, but I
> wouldn't want it merged with log_connections or logging of superuser
> connections.

Fair enough, we could have a log_replication_connections as a separate
one then? Or having one log_connections, one
log_replication_connections and one log_superuser_connections?


> Incidentally, I think ClientAuthentication_hook is sufficiently
> powerful to allow logging of superuser connections but no others, if
> someone wanted to write a contrib module.  That doesn't necessarily
> mean an in-core facility wouldn't be useful too, but it's at least
> worth thinking about using the hook.

Do we have an example of this hook somewhere already? If not, it could
be made into a useful example of that, perhaps?

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Replication logging

Robert Haas <robertmhaas@gmail.com> — 2011-01-17T15:03:46Z

On Mon, Jan 17, 2011 at 8:58 AM, Magnus Hagander <magnus@hagander.net> wrote:
>>>> What do you have in mind?
>>>
>>> Either having it controlled by log_connections, or perhaps have a
>>> log_highpriv_connections that controls replication *and* superuser, to
>>> be somewhat consistent.
>>
>> -1.  We could provide an option to turn this on and off, but I
>> wouldn't want it merged with log_connections or logging of superuser
>> connections.
>
> Fair enough, we could have a log_replication_connections as a separate
> one then? Or having one log_connections, one
> log_replication_connections and one log_superuser_connections?

log_replication_connections seems reasonable.  Not sure about
log_superuser_connections.

>> Incidentally, I think ClientAuthentication_hook is sufficiently
>> powerful to allow logging of superuser connections but no others, if
>> someone wanted to write a contrib module.  That doesn't necessarily
>> mean an in-core facility wouldn't be useful too, but it's at least
>> worth thinking about using the hook.
>
> Do we have an example of this hook somewhere already? If not, it could
> be made into a useful example of that, perhaps?

contrib/auth_delay

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: Replication logging

Magnus Hagander <magnus@hagander.net> — 2011-01-17T15:12:33Z

On Mon, Jan 17, 2011 at 16:03, Robert Haas <robertmhaas@gmail.com> wrote:
> On Mon, Jan 17, 2011 at 8:58 AM, Magnus Hagander <magnus@hagander.net> wrote:
>>>>> What do you have in mind?
>>>>
>>>> Either having it controlled by log_connections, or perhaps have a
>>>> log_highpriv_connections that controls replication *and* superuser, to
>>>> be somewhat consistent.
>>>
>>> -1.  We could provide an option to turn this on and off, but I
>>> wouldn't want it merged with log_connections or logging of superuser
>>> connections.
>>
>> Fair enough, we could have a log_replication_connections as a separate
>> one then? Or having one log_connections, one
>> log_replication_connections and one log_superuser_connections?
>
> log_replication_connections seems reasonable.  Not sure about
> log_superuser_connections.

So basically like this (see attachment).


>>> Incidentally, I think ClientAuthentication_hook is sufficiently
>>> powerful to allow logging of superuser connections but no others, if
>>> someone wanted to write a contrib module.  That doesn't necessarily
>>> mean an in-core facility wouldn't be useful too, but it's at least
>>> worth thinking about using the hook.
>>
>> Do we have an example of this hook somewhere already? If not, it could
>> be made into a useful example of that, perhaps?
>
> contrib/auth_delay

Hmm, ok, so not that then :-)


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Replication logging

Robert Haas <robertmhaas@gmail.com> — 2011-01-17T15:31:59Z

On Mon, Jan 17, 2011 at 10:12 AM, Magnus Hagander <magnus@hagander.net> wrote:
> On Mon, Jan 17, 2011 at 16:03, Robert Haas <robertmhaas@gmail.com> wrote:
>> On Mon, Jan 17, 2011 at 8:58 AM, Magnus Hagander <magnus@hagander.net> wrote:
>>>>>> What do you have in mind?
>>>>>
>>>>> Either having it controlled by log_connections, or perhaps have a
>>>>> log_highpriv_connections that controls replication *and* superuser, to
>>>>> be somewhat consistent.
>>>>
>>>> -1.  We could provide an option to turn this on and off, but I
>>>> wouldn't want it merged with log_connections or logging of superuser
>>>> connections.
>>>
>>> Fair enough, we could have a log_replication_connections as a separate
>>> one then? Or having one log_connections, one
>>> log_replication_connections and one log_superuser_connections?
>>
>> log_replication_connections seems reasonable.  Not sure about
>> log_superuser_connections.
>
> So basically like this (see attachment).

Yeah.  Although maybe we should take this opportunity to eliminate the
funky capitalization of Log_connections.

>>> Do we have an example of this hook somewhere already? If not, it could
>>> be made into a useful example of that, perhaps?
>>
>> contrib/auth_delay
>
> Hmm, ok, so not that then :-)

Doesn't preclude this.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: Replication logging

Magnus Hagander <magnus@hagander.net> — 2011-01-17T15:38:40Z

On Mon, Jan 17, 2011 at 16:31, Robert Haas <robertmhaas@gmail.com> wrote:
> On Mon, Jan 17, 2011 at 10:12 AM, Magnus Hagander <magnus@hagander.net> wrote:
>> On Mon, Jan 17, 2011 at 16:03, Robert Haas <robertmhaas@gmail.com> wrote:
>>> On Mon, Jan 17, 2011 at 8:58 AM, Magnus Hagander <magnus@hagander.net> wrote:
>>>>>>> What do you have in mind?
>>>>>>
>>>>>> Either having it controlled by log_connections, or perhaps have a
>>>>>> log_highpriv_connections that controls replication *and* superuser, to
>>>>>> be somewhat consistent.
>>>>>
>>>>> -1.  We could provide an option to turn this on and off, but I
>>>>> wouldn't want it merged with log_connections or logging of superuser
>>>>> connections.
>>>>
>>>> Fair enough, we could have a log_replication_connections as a separate
>>>> one then? Or having one log_connections, one
>>>> log_replication_connections and one log_superuser_connections?
>>>
>>> log_replication_connections seems reasonable.  Not sure about
>>> log_superuser_connections.
>>
>> So basically like this (see attachment).
>
> Yeah.  Although maybe we should take this opportunity to eliminate the
> funky capitalization of Log_connections.

We have that on several other variables as well, I'd rather see that
as a separate thing to change. But is it worth it, wrt it breaking
back-patchability?

Before I go ahead and commit the part that adds
log_replication_connections, anybody else want to object to the idea?
;)


>>>> Do we have an example of this hook somewhere already? If not, it could
>>>> be made into a useful example of that, perhaps?
>>>
>>> contrib/auth_delay
>>
>> Hmm, ok, so not that then :-)
>
> Doesn't preclude this.

Nope, but also doesn't make it the second reason to do it :-) I don't
personally have the itch to go write it, but if somebody does I can
always volunteer to review it...

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Replication logging

Tom Lane <tgl@sss.pgh.pa.us> — 2011-01-17T16:46:04Z

Magnus Hagander <magnus@hagander.net> writes:
> Before I go ahead and commit the part that adds
> log_replication_connections, anybody else want to object to the idea?

I think it'd make more sense just to say that replication connections
are subject to the same log_connections rule as others.  An extra GUC
for this is surely overkill.

			regards, tom lane

Re: Replication logging

Magnus Hagander <magnus@hagander.net> — 2011-01-17T16:58:03Z

On Mon, Jan 17, 2011 at 17:46, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Magnus Hagander <magnus@hagander.net> writes:
>> Before I go ahead and commit the part that adds
>> log_replication_connections, anybody else want to object to the idea?
>
> I think it'd make more sense just to say that replication connections
> are subject to the same log_connections rule as others.  An extra GUC
> for this is surely overkill.

I thought so, but Robert didn't agree. And given that things are the
way they are, clearly somebody else didn't agree as well - though I've
been unable to locate the original discussion if there was one.


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Replication logging

Tom Lane <tgl@sss.pgh.pa.us> — 2011-01-17T18:57:13Z

Magnus Hagander <magnus@hagander.net> writes:
> On Mon, Jan 17, 2011 at 17:46, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I think it'd make more sense just to say that replication connections
>> are subject to the same log_connections rule as others. An extra GUC
>> for this is surely overkill.

> I thought so, but Robert didn't agree. And given that things are the
> way they are, clearly somebody else didn't agree as well - though I've
> been unable to locate the original discussion if there was one.

The existing behavior dates from here:
http://archives.postgresql.org/pgsql-committers/2010-03/msg00245.php

As best I can tell there was no preceding discussion, just Simon
unilaterally deciding that this logging was required for debugging
purposes.  (There is a followup thread in -hackers arguing about the
message wording, but nobody questioned whether it should come out
unconditionally.)

I'm of the opinion that the correct way of "lowering in later releases"
is to make the messages obey Log_connections.  The "needed for debug"
argument seems mighty weak to me even for the time, and surely falls
down now.

			regards, tom lane

Re: Replication logging

Robert Haas <robertmhaas@gmail.com> — 2011-01-17T19:04:58Z

On Mon, Jan 17, 2011 at 1:57 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Magnus Hagander <magnus@hagander.net> writes:
>> On Mon, Jan 17, 2011 at 17:46, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> I think it'd make more sense just to say that replication connections
>>> are subject to the same log_connections rule as others.  An extra GUC
>>> for this is surely overkill.
>
>> I thought so, but Robert didn't agree. And given that things are the
>> way they are, clearly somebody else didn't agree as well - though I've
>> been unable to locate the original discussion if there was one.
>
> The existing behavior dates from here:
> http://archives.postgresql.org/pgsql-committers/2010-03/msg00245.php
>
> As best I can tell there was no preceding discussion, just Simon
> unilaterally deciding that this logging was required for debugging
> purposes.  (There is a followup thread in -hackers arguing about the
> message wording, but nobody questioned whether it should come out
> unconditionally.)
>
> I'm of the opinion that the correct way of "lowering in later releases"
> is to make the messages obey Log_connections.  The "needed for debug"
> argument seems mighty weak to me even for the time, and surely falls
> down now.

On a busy system, you could have a pretty high rate of messages
spewing forth for regular connections - that's a lot to wade through
if all you really want to see are the replication connections, which
should be much lower volume.  But I guess now that we have
pg_stat_replication it's a little easier to see the status of
replication anyway.  On the whole I've found the default setting here
very pleasant, so I'm reluctant to change it, but it sounds like I
might be out-voted.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: Replication logging

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> — 2011-01-18T07:15:52Z

On 17.01.2011 21:04, Robert Haas wrote:
> On Mon, Jan 17, 2011 at 1:57 PM, Tom Lane<tgl@sss.pgh.pa.us>  wrote:
>> I'm of the opinion that the correct way of "lowering in later releases"
>> is to make the messages obey Log_connections.  The "needed for debug"
>> argument seems mighty weak to me even for the time, and surely falls
>> down now.
>
> On a busy system, you could have a pretty high rate of messages
> spewing forth for regular connections - that's a lot to wade through
> if all you really want to see are the replication connections, which
> should be much lower volume.  But I guess now that we have
> pg_stat_replication it's a little easier to see the status of
> replication anyway.  On the whole I've found the default setting here
> very pleasant, so I'm reluctant to change it, but it sounds like I
> might be out-voted.

I also find it weird that incoming replication connections are logged by 
default. In the standby, we also log "streaming replication successfully 
connected to primary", which serves much of the same debugging purpose. 
That standby-side message is ok, we have a tradition of being more 
verbose during recovery, we also emit the "restored log file \"%s\" from 
archive" message for every WAL segment restored from archive for example.

We could turn log_connections into an enum, like log_statement:

log_connections = 'none'	# none, replication, regular, all

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

Re: Replication logging

Fujii Masao <masao.fujii@gmail.com> — 2011-01-18T07:21:35Z

On Tue, Jan 18, 2011 at 4:15 PM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:
> I also find it weird that incoming replication connections are logged by
> default. In the standby, we also log "streaming replication successfully
> connected to primary", which serves much of the same debugging purpose. That
> standby-side message is ok, we have a tradition of being more verbose during
> recovery, we also emit the "restored log file \"%s\" from archive" message
> for every WAL segment restored from archive for example.
>
> We could turn log_connections into an enum, like log_statement:
>
> log_connections = 'none'        # none, replication, regular, all

+1

We should treat log_disconnections the same?

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

Re: Replication logging

Magnus Hagander <magnus@hagander.net> — 2011-01-18T08:17:52Z

On Tue, Jan 18, 2011 at 08:21, Fujii Masao <masao.fujii@gmail.com> wrote:
> On Tue, Jan 18, 2011 at 4:15 PM, Heikki Linnakangas
> <heikki.linnakangas@enterprisedb.com> wrote:
>> I also find it weird that incoming replication connections are logged by
>> default. In the standby, we also log "streaming replication successfully
>> connected to primary", which serves much of the same debugging purpose. That
>> standby-side message is ok, we have a tradition of being more verbose during
>> recovery, we also emit the "restored log file \"%s\" from archive" message
>> for every WAL segment restored from archive for example.
>>
>> We could turn log_connections into an enum, like log_statement:
>>
>> log_connections = 'none'        # none, replication, regular, all

It almost seems overkill, but probably less so than a completely new guc :)


> We should treat log_disconnections the same?

We could keep it a boolean, but then only log disconnections for the
cases that are mentioned in log_connections?

It doesn't make sense to log disconnection for a connection we didn't
log the connection for...


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Replication logging

Fujii Masao <masao.fujii@gmail.com> — 2011-01-18T09:56:49Z

On Tue, Jan 18, 2011 at 5:17 PM, Magnus Hagander <magnus@hagander.net> wrote:
>> We should treat log_disconnections the same?
>
> We could keep it a boolean, but then only log disconnections for the
> cases that are mentioned in log_connections?
>
> It doesn't make sense to log disconnection for a connection we didn't
> log the connection for...

Maybe true. But, at least for me, it's more intuitive to provide both as
enum parameters.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

Re: Replication logging

Magnus Hagander <magnus@hagander.net> — 2011-01-18T15:42:38Z

On Tue, Jan 18, 2011 at 10:56, Fujii Masao <masao.fujii@gmail.com> wrote:
> On Tue, Jan 18, 2011 at 5:17 PM, Magnus Hagander <magnus@hagander.net> wrote:
>>> We should treat log_disconnections the same?
>>
>> We could keep it a boolean, but then only log disconnections for the
>> cases that are mentioned in log_connections?
>>
>> It doesn't make sense to log disconnection for a connection we didn't
>> log the connection for...
>
> Maybe true. But, at least for me, it's more intuitive to provide both as
> enum parameters.

Is there *any* usecase for setting them differently though? (other
than connections being <something> and disconnectoins being <none>?)
If not, aren't we just encouraging people to configure in a way that
makes no sense?

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Replication logging

Tom Lane <tgl@sss.pgh.pa.us> — 2011-01-18T15:57:35Z

Magnus Hagander <magnus@hagander.net> writes:
> Is there *any* usecase for setting them differently though?

I can't believe we're still engaged in painting this bikeshed.  Let's
just control it off log_connections and have done.

BTW, what about log_disconnections --- does a walsender emit a message
according to that?  If not, why not?  If we go through with something
fancy on the connection side, are we going to invent the same extra
complexity for log_disconnections?  And if we do, what happens when
they're set inconsistently?

			regards, tom lane

Re: Replication logging

Robert Haas <robertmhaas@gmail.com> — 2011-01-18T16:33:16Z

On Tue, Jan 18, 2011 at 2:15 AM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:
> I also find it weird that incoming replication connections are logged by
> default. In the standby, we also log "streaming replication successfully
> connected to primary", which serves much of the same debugging purpose.

Oh, good point.  All right, I withdraw my objection.  Let's just make
it all controlled by log_connections and go home.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: Replication logging

Magnus Hagander <magnus@hagander.net> — 2011-01-18T19:03:58Z

On Tue, Jan 18, 2011 at 17:33, Robert Haas <robertmhaas@gmail.com> wrote:
> On Tue, Jan 18, 2011 at 2:15 AM, Heikki Linnakangas
> <heikki.linnakangas@enterprisedb.com> wrote:
>> I also find it weird that incoming replication connections are logged by
>> default. In the standby, we also log "streaming replication successfully
>> connected to primary", which serves much of the same debugging purpose.
>
> Oh, good point.  All right, I withdraw my objection.  Let's just make
> it all controlled by log_connections and go home.

Done.

Oh, and the proper answer is yellow. Everybody knows that.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Replication logging

Josh Berkus <josh@agliodbs.com> — 2011-01-18T19:32:35Z

All,

Just speaking as someone who does a lot of log-crunching for performance
review, I don't find having a separate log_connections option for
replication terribly useful.  It's easy enough for me to just log all
connections and filter for the type of connections I want.

Even in cases where there's a ton of connection activity (e.g. PHP
webapp without a connection pool) logging all connections still doesn't
generate that many MB of output, in my experience.

-- 
                                  -- Josh Berkus
                                     PostgreSQL Experts Inc.
                                     http://www.pgexperts.com

Re: Replication logging

Simon Riggs <simon@2ndquadrant.com> — 2011-01-19T10:22:00Z

On Tue, 2011-01-18 at 10:57 -0500, Tom Lane wrote:
> Magnus Hagander <magnus@hagander.net> writes:
> > Is there *any* usecase for setting them differently though?
> 
> I can't believe we're still engaged in painting this bikeshed.  Let's
> just control it off log_connections and have done.

Yes, this is a waste of time. Leave it as is because its there for a
very good reason, as Robert already explained.

The code was put in explicitly because debugging replication connections
is quite important and prior to that addition, wasn't easy. It's a very
separate thing from logging the hundreds/thousands of other connections
on the system.

I don't really care about neatness of code, or neatness of parameters. I
want to be able to confirm the details of the connection.
pg_stat_replication is dynamic, not a historical record of connections
and reconnections.

How else will you diagnose an erratic network, or an accidental change
of authority? Replication is so important it isn't worth tidying away a
couple of lines of log. My objective is to make replication work, not to
reduce the size of the average log file by 1 or 2 lines.

I'm particularly concerned that people make such changes too quickly.
There are many things in this area of code that need changing, but also
many more that do not. If we are to move forwards we need to avoid going
one step forwards, one step back.

-- 
 Simon Riggs           http://www.2ndQuadrant.com/books/
 PostgreSQL Development, 24x7 Support, Training and Services

Re: Replication logging

Bruce Momjian <bruce@momjian.us> — 2011-01-19T16:59:14Z

Simon Riggs wrote:
> On Tue, 2011-01-18 at 10:57 -0500, Tom Lane wrote:
> > Magnus Hagander <magnus@hagander.net> writes:
> > > Is there *any* usecase for setting them differently though?
> > 
> > I can't believe we're still engaged in painting this bikeshed.  Let's
> > just control it off log_connections and have done.
> 
> Yes, this is a waste of time. Leave it as is because its there for a
> very good reason, as Robert already explained.
> 
> The code was put in explicitly because debugging replication connections
> is quite important and prior to that addition, wasn't easy. It's a very
> separate thing from logging the hundreds/thousands of other connections
> on the system.
> 
> I don't really care about neatness of code, or neatness of parameters. I
> want to be able to confirm the details of the connection.
> pg_stat_replication is dynamic, not a historical record of connections
> and reconnections.
> 
> How else will you diagnose an erratic network, or an accidental change
> of authority? Replication is so important it isn't worth tidying away a
> couple of lines of log. My objective is to make replication work, not to
> reduce the size of the average log file by 1 or 2 lines.
> 
> I'm particularly concerned that people make such changes too quickly.
> There are many things in this area of code that need changing, but also
> many more that do not. If we are to move forwards we need to avoid going
> one step forwards, one step back.

There were enough people who wanted a change that we went ahead and did
it --- if there was lack of agreement, we would have delayed longer.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + It's impossible for everything to be true. +

Re: Replication logging

Tom Lane <tgl@sss.pgh.pa.us> — 2011-01-19T17:05:39Z

Bruce Momjian <bruce@momjian.us> writes:
> Simon Riggs wrote:
>> I'm particularly concerned that people make such changes too quickly.
>> There are many things in this area of code that need changing, but also
>> many more that do not. If we are to move forwards we need to avoid going
>> one step forwards, one step back.

> There were enough people who wanted a change that we went ahead and did
> it --- if there was lack of agreement, we would have delayed longer.

The real reason why we changed this is that nobody (except Simon) sees
a situation where unconditional logging of successful replication
connections is especially helpful.  If you were trying to diagnose a
problem you would more likely need to know about *failed* connections,
but the code that was in there didn't provide that.  At least not unless
you turned on log_connections ...

			regards, tom lane

Re: Replication logging

Bruce Momjian <bruce@momjian.us> — 2011-01-19T17:15:50Z

Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > Simon Riggs wrote:
> >> I'm particularly concerned that people make such changes too quickly.
> >> There are many things in this area of code that need changing, but also
> >> many more that do not. If we are to move forwards we need to avoid going
> >> one step forwards, one step back.
> 
> > There were enough people who wanted a change that we went ahead and did
> > it --- if there was lack of agreement, we would have delayed longer.
> 
> The real reason why we changed this is that nobody (except Simon) sees
> a situation where unconditional logging of successful replication
> connections is especially helpful.  If you were trying to diagnose a
> problem you would more likely need to know about *failed* connections,
> but the code that was in there didn't provide that.  At least not unless
> you turned on log_connections ...

Yep.  

Simon, I understand you being disappointed you could not champion your
cause during the discussion, but you can still try to sway people that
the original code was appropriate.  

Based on the feedback from the group, it seemed unlikely you would be
able to sway a significant number of people so we just went ahead and
made the change.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + It's impossible for everything to be true. +