Re: Log file to sql database

KuroiNeko <evpopkov@carrier.kiev.ua> writes:

> > I am  not sure  I understand  the implications  of the  piping it  to the
> > insert
> > routine.
> 
>  It's a double-edged sword. Invoking it is  as simple as adding a line to a
> conf file,  while developing it  may turn out to  be tricky. At  least make
> sure you examine output of ps ax to see how much CPU time it uses.
>  And if  you pipe  it to a  script, you'll have  an interpreter  loaded and
> sitting there wasting your RAM.

Also, if your pipe reader slows down too much, syslogd will start
blocking on the pipe, which will slow down the whole system (syslogd
generally isn't multi-threaded).  So I think it'd be much safer to
have a flat text logfile and have your program read from that (using
similar code to 'tail -f', as was suggested earlier).  That way if you
fall behind during a busy period you can catch up once things ease up, 
and syslogd doesn't kill your whole system.

As for the interpreter issue, well, I'd rather spend the memory and
use a language that is designed for string-slinging.  Using Perl or
Python to parse things is *so* much easier than C (even with a good
regex library available).

As always, just MHO...

-Doug