Thread

1. Re: [HACKERS] RFC: Industrial-strength logging (long message)

   Tom Lane <tgl@sss.pgh.pa.us> — 1999-10-24T16:15:32Z

   Tim Holloway <mtsinc@southeast.net> writes:
   >> Note that logging into a table is harder than you might think.
   
   > I guess I should have mentioned - at least in its initial incarnation,
   > cowardice forbids me to attempt reading or writing PostgreSQL tables
   > directly. The logfile design is designed to be text and
   > customizable. If one of those custom formats just happens to look like
   > loadable data, well..... :)
   
   Yeah, someone else suggested writing to a textfile and then having a
   cron task or something like that load the data into a table later on.
   That seems workable, but you'd need some answer to the following
   problem.  Suppose that for some reason (like trying to diagnose a
   transient problem) the log level is currently set high enough so that
   every "insert" command generates a log entry.  The appointed hour
   comes 'round and your cron task fires off.  Each time it copies data
   out of the logfile into the database, it is itself adding at least one
   more entry to the logfile.  Can you say "infinite loop"?
   
   I can think of a couple of possible workarounds, but the one that seems
   most natural is to let the logging task override the system-wide logging
   level and set its own log level to something low.  That ties right in
   with your followup comment:
   
   > For simple things like log levels, though, I'd still like feedback on
   > desirablility and feasibility of altering basic logging options though
   > (authorized!) frontends.
   
   I think you were thinking here of altering the system-wide level through
   a frontend command, but what I'm envisioning is allowing an SQL client
   to alter the log level for its own particular backend *without* any
   system-wide effects.
   
   Even that ability might need to be restricted to suitably-privileged
   users, else it could be used to "fly under the radar" of an admin who
   was using logging for security purposes.  Perhaps I'm being overly
   paranoid, though.  There are probably only a few message types that
   might be of interest for security purposes, so maybe we could define the
   filtering commands in such a way that those messages are not disablable
   from a client.  Anyone else have strong feelings about this?
   
   >> No, it should definitely not be part of pg_hba.conf, it should
   >> be a separate configuration file.  (pg_hba.conf has a syntax too
   >> simple to be easily extensible.)
   
   > Of more concern to me was that I THINK I saw pg_hba.conf being
   > rescanned whenever security was tested.
   
   That's true, pg_hba.conf is currently reread on each connection attempt.
   We probably ought to try to avoid that... but in any case I think there
   are obvious security reasons for keeping access authorization info
   strictly separate from other configuration data.
   
   > Good point, but it was the second item on the message
   > override line:
   
   > 101 INFO "Server started"
   >      A
   > -----+
   
   Oops, I missed that...
   
   			regards, tom lane
   
   

2. Re: [HACKERS] RFC: Industrial-strength logging (long message)

   Tim Holloway <mtsinc@southeast.net> — 1999-10-24T17:48:20Z

   
   Tom Lane wrote:
   >
   > Tim Holloway <mtsinc@southeast.net> writes:
   > >> Note that logging into a table is harder than you might think.
   >
   > > I guess I should have mentioned - at least in its initial incarnation,
   > > cowardice forbids me to attempt reading or writing PostgreSQL tables
   > > directly. The logfile design is designed to be text and
   > > customizable. If one of those custom formats just happens to look like
   > > loadable data, well..... :)
   >
   > Yeah, someone else suggested writing to a textfile and then having a
   > cron task or something like that load the data into a table later on.
   > That seems workable, but you'd need some answer to the following
   > problem.  Suppose that for some reason (like trying to diagnose a
   > transient problem) the log level is currently set high enough so that
   > every "insert" command generates a log entry.  The appointed hour
   > comes 'round and your cron task fires off.  Each time it copies data
   > out of the logfile into the database, it is itself adding at least one
   > more entry to the logfile.  Can you say "infinite loop"?
   
   You noticed that too, eh? You might want to take a look at the archived
   psql-admin postings about the middle of last week. Since I'm working on the
   premise that all log files are text files and there's already been the desire
   expressed that they be rotatable, it's simplest to piggyback the load function
   onto rotation: start a new file and load the prior one. It introduces
   some latency into the log tables (forcing rotation can obviously cure this),
   but should eliminate the log recursion by deferring the entries.
   Hmmmm. Maybe the initial log filter WON'T be just a stub!
   
   >
   > > For simple things like log levels, though, I'd still like feedback on
   > > desirablility and feasibility of altering basic logging options though
   > > (authorized!) frontends.
   >
   > I think you were thinking here of altering the system-wide level through
   > a frontend command, but what I'm envisioning is allowing an SQL client
   > to alter the log level for its own particular backend *without* any
   > system-wide effects.
   >
   > Even that ability might need to be restricted to suitably-privileged
   > users, else it could be used to "fly under the radar" of an admin who
   > was using logging for security purposes.  Perhaps I'm being overly
   > paranoid, though.  There are probably only a few message types that
   > might be of interest for security purposes, so maybe we could define the
   > filtering commands in such a way that those messages are not disablable
   > from a client.  Anyone else have strong feelings about this?
   
   I seem to read a desire to log frontend action in what you're saying.
   I guess I should define my ambitions. Initially, at least, all I'm trying
   to log is server activity, and that from an administrative point of view.
   I don't plan to subsume the debugging system, because:
   
   	1. IMHO, it works fine as is (excepting the lack of timestamping)
   	2. The debugging messages weren't designed to fit the constraints of
   	the logger. That would require reworking dozens of messages all over
   	the program. I'd almost certainly break something critical.
   
   Of course, the line between event logging and debugging is pretty fuzzy and
   apt to change, depending on what you need at the moment.
   
   I didn't consider logging as related to front-ends, since: they're more
   of a programmer's problem; there exist a multitude of them, and some -
   like ODBC - already have their own logging. I'm open to suggestion, though
   I think that's too big a bite to chew just yet.