Thread

1. Re: [HACKERS] RFC: Industrial-strength logging (long message)

   Tom Lane <tgl@sss.pgh.pa.us> — 1999-10-23T17:25:05Z

   Tim Holloway <mtsinc@southeast.net> writes:
   > Request For Comments:  Towards an industrial-strength
   > logging facility
   
   Sounds pretty good overall.
   
   > This depends. I like a console-style listing, as my needs are
   > simple. Others would prefer that the log be itself a database.
   
   Note that logging into a table is harder than you might think.
   For example: (1) The postmaster cannot touch tables at all, so
   no events detected in the postmaster could be logged that way.
   (2) No events detected during a transaction that ultimately
   aborts will be successfully logged.  (3) Logging events such as
   "server failure" would be quite risky --- if the server has
   suffered internal state corruption then it could manage to
   corrupt the log table entirely while it's trying to add its
   last-dying-breath report.
   
   Fortunately none of these problems apply to stderr, syslog,
   or plain-text-file reporting channels.
   
   > There are 3 types of information in the logging
   > configuration file (which may, but likely won't, be part of
   > pg_hba.conf)
   
   No, it should definitely not be part of pg_hba.conf, it should
   be a separate configuration file.  (pg_hba.conf has a syntax too
   simple to be easily extensible.)
   
   Another possibility is to keep the config info in a system table, but
   that would have a number of drawbacks (the postmaster cannot read
   tables, nor can a backend that's just starting up and hasn't finished
   initialization).  On the whole, a plain text file in the database
   directory is probably the best bet.
   
   > Logging info is read at startup. There may
   > exist signals that cause it to be reread, but not just yet.
   
   There MUST exist a way to alter the logging level on-the-fly;
   IMHO this is a rock bottom, non negotiable requirement.
   A production system can't restart the postmaster just to tweak
   the logging level up or down for investigation of a problem.
   
   Whether it's a signal or something else remains to be determined.
   We have pretty nearly used up all the available signal numbers :-(.
   I suppose that whichever signal is currently used to trigger
   rereading of the pg_options configuration file could also trigger
   re-reading of the logging config file.
   
   > To avoid potential loss
   > of critical info, any message not explicitly routed at least
   > once gets reported on the default channel - stderr/syslog,
   > unless otherwise configured.
   
   Hmm, so I'd have to explicitly discard every message I didn't want to
   hear about?  I think that "forced display" like this should only happen
   for high-severity messages, not for routine junk.  There doesn't seem to
   be any notion of message importance in your design, but I think there
   should be.  Most people would probably prefer to filter on an importance
   level, only occasionally resorting to calling out specific message types.
   
   > Especially of interest is what the shape of the config file should be.
   > Is the the pseudo-HTML format shown good?
   
   You could do worse than to borrow BIND's syntax for log control.
   
   			regards, tom lane
   
   

2. Re: [ADMIN] Re: [HACKERS] RFC: Industrial-strength logging (long message)

   Aaron J. Seigo <aaron@gtv.ca> — 1999-10-23T17:54:40Z

   hi...
   
   > > This depends. I like a console-style listing, as my needs are
   > > simple. Others would prefer that the log be itself a database.
   > 
   > Note that logging into a table is harder than you might think.
   
   unless i misunderstand, the concept is to design the logs such that it is
   trivial to convert them into a database, even including tools to do this, not
   to actually create a database on the fly.
   
    i'm also supportive of including tools to run standard reports out-of-the-box
   on these logs. using pgsql to massage its own logs is pretty sexy, imo =)
   
   > No, it should definitely not be part of pg_hba.conf, it should
   > be a separate configuration file.  (pg_hba.conf has a syntax too
   > simple to be easily extensible.)
   > initialization).  On the whole, a plain text file in the database
   > directory is probably the best bet.
   
   agreed...
   
   > There MUST exist a way to alter the logging level on-the-fly;
   > IMHO this is a rock bottom, non negotiable requirement.
   
   whilst i don't think this is MUST, it is EXTREMELY desirable and would make the
   logging actually useful for large installations =)
   
   > Whether it's a signal or something else remains to be determined.
   > We have pretty nearly used up all the available signal numbers :-(.
   > I suppose that whichever signal is currently used to trigger
   > rereading of the pg_options configuration file could also trigger
   > re-reading of the logging config file.
   
   why not use pg_options for logging cofig?
   
   > Hmm, so I'd have to explicitly discard every message I didn't want to
   > hear about?  I think that "forced display" like this should only happen
   > for high-severity messages, not for routine junk.  There doesn't seem to
   > be any notion of message importance in your design, but I think there
   > should be.  Most people would probably prefer to filter on an importance
   > level, only occasionally resorting to calling out specific message types.
   
   systems i've dealt with in the past prioritize (as you mentioned) on a numeric
   scale to reflect "importance" and the logging level is by default set at a
   certain "height"... increasing logging is as easy as changing the threshold..
   this has been effective in the past...
   
   the only problem with ONLY relying on thresholds is that its a very coarse
   grain method... so when you request a lower level of logging, you often get
   inundated with all SORTS of stuff you don't really care/want...
   
   a hybridization between the two approaches might be best.. e.g., each type of
   message gets assigned a "criticality", between 1 and 10 perhaps. with 1 being
   most critical, and 10 being least..  therefore, the higher you set your logging
   level, the more messages you get (logical?)
   
   the default behaviour for each level (1..10) is to let everything though on
   that level. but you can apply filters to limit the output...
   so you get the ability to define grossly with thresholds (1..10) and finely
   (and optionally) with message filters...
   
   > You could do worse than to borrow BIND's syntax for log control.
   
   much worse. =)
   
   -- 
   Aaron J. Seigo
   Sys Admin
   
   

3. Re: [ADMIN] Re: [HACKERS] RFC: Industrial-strength logging (long message)

   Marc G. Fournier <scrappy@hub.org> — 1999-10-23T19:23:57Z

   On Sat, 23 Oct 1999, Aaron J. Seigo wrote:
   
   > > There MUST exist a way to alter the logging level on-the-fly;
   > > IMHO this is a rock bottom, non negotiable requirement.
   > 
   > whilst i don't think this is MUST, it is EXTREMELY desirable and would make the
   > logging actually useful for large installations =)
   
   Let's re-iterate Tom here: There MUST exist a way ... someone *MUST* be
   able to change their configuration without having to physically stop/start
   the server to affect the changes ...
   
   Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
   Systems Administrator @ hub.org 
   primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 
   
   
   

4. Re: [HACKERS] RFC: Industrial-strength logging (long message)

   Tim Holloway <mtsinc@southeast.net> — 1999-10-24T00:47:13Z

   
   Tom Lane wrote:
   > 
   > Tim Holloway <mtsinc@southeast.net> writes:
   > > Request For Comments:  Towards an industrial-strength
   > > logging facility
   > 
   > Sounds pretty good overall.
   > 
   > > This depends. I like a console-style listing, as my needs are
   > > simple. Others would prefer that the log be itself a database.
   > 
   > Note that logging into a table is harder than you might think.
   > For example: (1) The postmaster cannot touch tables at all, so
   > no events detected in the postmaster could be logged that way.
   > (2) No events detected during a transaction that ultimately
   > aborts will be successfully logged.  (3) Logging events such as
   > "server failure" would be quite risky --- if the server has
   > suffered internal state corruption then it could manage to
   > corrupt the log table entirely while it's trying to add its
   > last-dying-breath report.
   > 
   > Fortunately none of these problems apply to stderr, syslog,
   > or plain-text-file reporting channels.
   
   Thanks, Tom! I'l file this collection of wisdom to help keep
   me on the straight and
   narrow. I guess I should have mentioned - at least in its
   initial incarnation, cowardice
   forbids me to attempt reading or writing PostgreSQL tables
   directly. The logfile design is
   designed to be text and customizable. If one of those custom
   formats just happens to look
   like loadable data, well..... :) 
   
   BTW, cowardice also forbids me to attempt message filtering
   except by message ID or severity
   just yet (no "log all requests from Clevleand to channel 2"
   stuff). I will try to provide a
   stub for the adventurous, though. For everyone else, there's
   Perl.
   
   > > There are 3 types of information in the logging
   > > configuration file (which may, but likely won't, be part of
   > > pg_hba.conf)
   > 
   > No, it should definitely not be part of pg_hba.conf, it should
   > be a separate configuration file.  (pg_hba.conf has a syntax too
   > simple to be easily extensible.)
   
   Of more concern to me was that I THINK I saw pg_hba.conf
   being rescanned whenever security
   was tested. I don't want to slow that down with a lot of
   "one-time" (see below) data.
   > 
   > Another possibility is to keep the config info in a system table, but
   > that would have a number of drawbacks (the postmaster cannot read
   > tables, nor can a backend that's just starting up and hasn't finished
   > initialization).  On the whole, a plain text file in the database
   > directory is probably the best bet.
   
   I think so too -- you just reinforced my feelings. There's
   no intrinsic
   benefit, since the error messages and channel definition get
   compiled
   into memory, anyhow.
   > 
   > > Logging info is read at startup. There may
   > > exist signals that cause it to be reread, but not just yet.
   > 
   > There MUST exist a way to alter the logging level on-the-fly;
   > IMHO this is a rock bottom, non negotiable requirement.
   > A production system can't restart the postmaster just to tweak
   > the logging level up or down for investigation of a problem.
   
   OK, I'm convinced!
   > 
   > Whether it's a signal or something else remains to be determined.
   > We have pretty nearly used up all the available signal numbers :-(.
   > I suppose that whichever signal is currently used to trigger
   > rereading of the pg_options configuration file could also trigger
   > re-reading of the logging config file.
   
   How about via psql or other facilities passing a message
   packet?
   Can you think of any cases where this would fail? BETTER
   YET!
   Is there any reason whay pg_options cannot be extended? It
   seems like
   a natural fit to me - the only reason I didn't suggest it
   originally was that
   it's been so low-key, I forgot it was there!
   
   > 
   > > To avoid potential loss
   > > of critical info, any message not explicitly routed at least
   > > once gets reported on the default channel - stderr/syslog,
   > > unless otherwise configured.
   > 
   > Hmm, so I'd have to explicitly discard every message I didn't want to
   > hear about?  I think that "forced display" like this should only happen
   > for high-severity messages, not for routine junk.  There doesn't seem to
   > be any notion of message importance in your design, but I think there
   > should be.  Most people would probably prefer to filter on an importance
   > level, only occasionally resorting to calling out specific message types.
   
   Good point, but it was the second item on the message
   override line:
   
   101 INFO "Server started"
        A
   -----+
   
   The intent was actually more to ensure that if a new release
   added new messages, they
   wouldn't suddenly pop up in places they'd cause the receptor
   to get indigestion (e.g. table loader)
   or have critical messages get lost entirely. I did the
   pseudocode for lossless message routing
   today -- adding a dropout threshold doesn't look like a
   major problem.
   > 
   > > Especially of interest is what the shape of the config file should be.
   > > Is the the pseudo-HTML format shown good?
   > 
   > You could do worse than to borrow BIND's syntax for log control.
   
   *I* like it (I must - I stole almost everything else from
   there!). That's what I meant
   by "is a "C" format good?". It works well as an extension to
   pg_options. Just wanted to see what
   others would be most comfortable with.
   
   Again, thanks! This is a big help!
   
      Tim Holloway
   
   

5. Re: [ADMIN] Re: [HACKERS] RFC: Industrial-strength logging (longmessage)

   Tim Holloway <mtsinc@southeast.net> — 1999-10-24T01:07:52Z

   
   The Hermit Hacker wrote:
   > 
   > On Sat, 23 Oct 1999, Aaron J. Seigo wrote:
   > 
   > > > There MUST exist a way to alter the logging level on-the-fly;
   > > > IMHO this is a rock bottom, non negotiable requirement.
   > >
   > > whilst i don't think this is MUST, it is EXTREMELY desirable and would make the
   > > logging actually useful for large installations =)
   > 
   > Let's re-iterate Tom here: There MUST exist a way ... someone *MUST* be
   > able to change their configuration without having to physically stop/start
   > the server to affect the changes ...
   > 
   > Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
   > Systems Administrator @ hub.org
   > primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org
   
   I think we have a consensus. Destroy and recreate logging
   data structures/tasks on receipt of
   suitable event.
   
   For simple things like log levels, though, I'd still like
   feedback on
   desirablility and feasibility of altering basic logging
   options though
   (authorized!) frontends. As a user, I get nervous when I
   have to thread
   my way past possibly-fragile unrelated items in a config
   file when I'm trying
   to do a panic diagnosis. As an administrator, I get even
   MORE nervous if one
   of the less careful people I know were to be entrusted with
   that task.
   
   Another possible mode of controlling what's logged is to
   assign mask bits to various
   classes of messaages and allow the administrator to alter
   the filter mask.
   Although, in truth, the channel design is pretty much the
   same thing.
   
       Tim Holloway
   
   

6. Re: [ADMIN] Re: [HACKERS] RFC: Industrial-strength logging (longmessage)

   Marc G. Fournier <scrappy@hub.org> — 1999-10-24T17:12:27Z

   Why not do something similar to what we are doing with pg_shadow?  If I
   remember the logic right, when you update pg_shadow, one ofits "steps" is
   to dump it to a text file so that postmaster can read it?  this should
   make it easy for one user/database to have one logging set, while another
   doesn' have it set at all...and should make it so that each database
   *should* theoretically log to different files/mechanisms?
   
   On Sat, 23 Oct 1999, Tim Holloway wrote:
   
   > 
   > 
   > The Hermit Hacker wrote:
   > > 
   > > On Sat, 23 Oct 1999, Aaron J. Seigo wrote:
   > > 
   > > > > There MUST exist a way to alter the logging level on-the-fly;
   > > > > IMHO this is a rock bottom, non negotiable requirement.
   > > >
   > > > whilst i don't think this is MUST, it is EXTREMELY desirable and would make the
   > > > logging actually useful for large installations =)
   > > 
   > > Let's re-iterate Tom here: There MUST exist a way ... someone *MUST* be
   > > able to change their configuration without having to physically stop/start
   > > the server to affect the changes ...
   > > 
   > > Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
   > > Systems Administrator @ hub.org
   > > primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org
   > 
   > I think we have a consensus. Destroy and recreate logging
   > data structures/tasks on receipt of
   > suitable event.
   > 
   > For simple things like log levels, though, I'd still like
   > feedback on
   > desirablility and feasibility of altering basic logging
   > options though
   > (authorized!) frontends. As a user, I get nervous when I
   > have to thread
   > my way past possibly-fragile unrelated items in a config
   > file when I'm trying
   > to do a panic diagnosis. As an administrator, I get even
   > MORE nervous if one
   > of the less careful people I know were to be entrusted with
   > that task.
   > 
   > Another possible mode of controlling what's logged is to
   > assign mask bits to various
   > classes of messaages and allow the administrator to alter
   > the filter mask.
   > Although, in truth, the channel design is pretty much the
   > same thing.
   > 
   >     Tim Holloway
   > 
   
   Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
   Systems Administrator @ hub.org 
   primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 
   
   
   

7. Re: [HACKERS] RFC: Industrial-strength logging

   Peter Eisentraut <peter_e@gmx.net> — 1999-10-24T20:15:51Z

   On Oct 23, Tim Holloway mentioned:
   
   > I think we have a consensus. Destroy and recreate logging data
   > structures/tasks on receipt of suitable event.
   > 
   > For simple things like log levels, though, I'd still like feedback on
   > desirablility and feasibility of altering basic logging options though
   > (authorized!) frontends. As a user, I get nervous when I have to
   > thread my way past possibly-fragile unrelated items in a config file
   > when I'm trying to do a panic diagnosis. As an administrator, I get
   > even MORE nervous if one of the less careful people I know were to be
   > entrusted with that task.
   
   What about
   SET LOGLEVEL TO <something>;
   SET LOGDETAIL TO <something>;
   or the like. You could use pg_shadow.usesuper as a security stipulation.
   Using something like a signal to do this is probably overkill, especially
   since there are hardly any left, and it's also infinitely less intuitive
   and flexible.
   
   	-Peter
   
   -- 
   Peter Eisentraut                  Sernanders vaeg 10:115
   peter_e@gmx.net                   75262 Uppsala
   http://yi.org/peter-e/            Sweden
   
   
   

8. Re: [HACKERS] RFC: Industrial-strength logging

   Aaron J. Seigo <aaron@gtv.ca> — 1999-10-25T21:50:59Z

   hi...
   
   > What about
   > SET LOGLEVEL TO <something>;
   > SET LOGDETAIL TO <something>;
   > or the like. You could use pg_shadow.usesuper as a security stipulation.
   > Using something like a signal to do this is probably overkill, especially
   > since there are hardly any left, and it's also infinitely less intuitive
   > and flexible.
   
   this would be done from psql? if so, here's a query i have: are there any plans
   to seperate the admin functions out of psql and into another seperate tool? 
   
   i have a queasyness with general users having access to a tool that can
   do admin takss (even if they supposedly don't have a superuser account).
   
   it also seems much cleaner to have admin in one tool and data interaction in
   another.
   
   am i off track here?
   
   -- 
   Aaron J. Seigo
   Sys Admin
   
   

9. Re: [HACKERS] RFC: Industrial-strength logging

   Marc G. Fournier <scrappy@hub.org> — 1999-10-26T01:01:05Z

   On Mon, 25 Oct 1999, Aaron J. Seigo wrote:
   
   > hi...
   > 
   > > What about
   > > SET LOGLEVEL TO <something>;
   > > SET LOGDETAIL TO <something>;
   > > or the like. You could use pg_shadow.usesuper as a security stipulation.
   > > Using something like a signal to do this is probably overkill, especially
   > > since there are hardly any left, and it's also infinitely less intuitive
   > > and flexible.
   > 
   > this would be done from psql? if so, here's a query i have: are there any plans
   > to seperate the admin functions out of psql and into another seperate tool? 
   > 
   > i have a queasyness with general users having access to a tool that can
   > do admin takss (even if they supposedly don't have a superuser account).
   
   There is no such thing, actually...all "admin commands" are seperate SQL
   queries...psql is merely one of many interfaces that allows one to talk to
   and send queries to the backend...what the backend then does with the
   query is where the security lies...
   
   Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
   Systems Administrator @ hub.org 
   primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 
   
   
   

10. Re: [ADMIN] Re: [HACKERS] RFC: Industrial-strength logging (longmessage)

    Bruce Momjian <maillist@candle.pha.pa.us> — 1999-10-26T05:27:54Z

    > I think we have a consensus. Destroy and recreate logging
    > data structures/tasks on receipt of
    > suitable event.
    > 
    > For simple things like log levels, though, I'd still like
    > feedback on
    > desirablility and feasibility of altering basic logging
    > options though
    > (authorized!) frontends. As a user, I get nervous when I
    > have to thread
    > my way past possibly-fragile unrelated items in a config
    > file when I'm trying
    > to do a panic diagnosis. As an administrator, I get even
    > MORE nervous if one
    > of the less careful people I know were to be entrusted with
    > that task.
    
    One more item I have not heard is that you can create virtual table that
    look like tables but return data about users, queries on SELECT.
    
    Informix does this.  Allows you to get info, without the need for
    storage.  Not good for every case, but an interesting idea sometimes.
    
    -- 
      Bruce Momjian                        |  http://www.op.net/~candle
      maillist@candle.pha.pa.us            |  (610) 853-3000
      +  If your life is a hard drive,     |  830 Blythe Avenue
      +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
    
    

11. Re: [ADMIN] Re: [HACKERS] RFC: Industrial-strength logging (longmessage)

    Hannu Krosing <hannu@tm.ee> — 1999-10-26T07:27:55Z

    Bruce Momjian wrote:
    > 
    > 
    > One more item I have not heard is that you can create virtual table that
    > look like tables but return data about users, queries on SELECT.
    > 
    > Informix does this.  Allows you to get info, without the need for
    > storage.  Not good for every case, but an interesting idea sometimes.
    > 
    
    This should come naturally after the function interface is updated
    to enable it to return cursors.
    
    A very desirable feature, but I'm not sure anyone is actually working on it.
    
    -------------------
    Hannu