Thread

1. Re: Using Threads?

   Myron Scott <mscott@sacadia.com> — 2001-01-02T15:43:55Z

   Karel Zak wrote:
   
    > On Tue, 2 Jan 2001, Myron Scott wrote:
    >
    >
    >> spinlocks rewritten to mutex_
    >> locktable uses sema_
    >> some cond_ in bufmgr.c
    >
    >
    > Interesting, have you some comperation between IPC PostgresSQl anf
    > your thread based PostgreSQL.
    >
    > Karel
   
   Yes, I did some comparisions but it is hard to
   make accurate evaluations on the data.  I basically
   did 1000 inserts from 7.0.2 and the modified verison
   from 8 simultaneous clients.  The original 7.0.2
   was faster by an an order of magnitude.  This needs to
   looked into more though. It was just a rough test b/c clients
   and server all running on the same machine (Ultra 10
   w/512MB RAM).
   
   I don't really know what the impact of changing some of
   the locking mechanisms is.
   
   On the one hand, there is alot of overhead associated
   with using TAO ORB as the fe<->be protocol.  The 7.0.2
   fe<->be is pretty efficient, TAO with IIOP not as much so.
   At the same time, using prepared statements when doing
   the same insert with different variables over and over
   cuts out re-parsing and planning the statement on every
   execute.
   
   Lastly,  I really didn't optimize my code at all.  There
   are some places where GetEnv() in called over and over
   to get the thread local variable where it should only
   be called once in the method and reused.
   
   Speed wasn't the motivation, I just wanted to see if threads
   and PostgreSQL could be done.
   
   Myron
   
   
   
   

2. Re: Using Threads?

   Alfred Perlstein <bright@wintelcom.net> — 2001-01-02T15:52:34Z

   * Myron Scott <mscott@sacadia.com> [010102 07:45] wrote:
   > 
   > Karel Zak wrote:
   > 
   >  > On Tue, 2 Jan 2001, Myron Scott wrote:
   >  >
   >  >
   >  >> spinlocks rewritten to mutex_
   >  >> locktable uses sema_
   >  >> some cond_ in bufmgr.c
   >  >
   >  >
   >  > Interesting, have you some comperation between IPC PostgresSQl anf
   >  > your thread based PostgreSQL.
   >  >
   >  > Karel
   > 
   > Yes, I did some comparisions but it is hard to
   > make accurate evaluations on the data.  I basically
   > did 1000 inserts from 7.0.2 and the modified verison
   > from 8 simultaneous clients.  The original 7.0.2
   > was faster by an an order of magnitude.  This needs to
   > looked into more though. It was just a rough test b/c clients
   > and server all running on the same machine (Ultra 10
   > w/512MB RAM).
   
   It's possible what you're seeing is the entire process
   wait for a disk IO to complete.
   
   I'm wondering, how many lwps does your system use?  Are all
   the threads bound to a single lwp or do you let the threads
   manager handle this all for you?
   
   > I don't really know what the impact of changing some of
   > the locking mechanisms is.
   
   heh. :)
   
   > On the one hand, there is alot of overhead associated
   > with using TAO ORB as the fe<->be protocol.  The 7.0.2
   > fe<->be is pretty efficient, TAO with IIOP not as much so.
   > At the same time, using prepared statements when doing
   > the same insert with different variables over and over
   > cuts out re-parsing and planning the statement on every
   > execute.
   > 
   > Lastly,  I really didn't optimize my code at all.  There
   > are some places where GetEnv() in called over and over
   > to get the thread local variable where it should only
   > be called once in the method and reused.
   > 
   > Speed wasn't the motivation, I just wanted to see if threads
   > and PostgreSQL could be done.
   
   Well it'd be better to see if threads+postgresql could be
   made faster than non-threaded postgresql.
   
   Do you see yourself with some extra free time in the upcommming
   weeks? :)
   
   -- 
   -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
   "I have the heart of a child; I keep it in a jar on my desk."
   
   

3. Re: Using Threads?

   Myron Scott <mscott@sacadia.com> — 2001-01-02T16:46:02Z

   
   Alfred Perlstein wrote:
   
   
   > 
   > It's possible what you're seeing is the entire process
   > wait for a disk IO to complete.
   > 
   > I'm wondering, how many lwps does your system use?  Are all
   > the threads bound to a single lwp or do you let the threads
   > manager handle this all for you?
   > 
   
   Yeah, I looked at this.  I have one thread per process that
   does all flushing of buffer pages at transaction commit.
   The client threads register buffer writes with this thread and
   wait for this thread to complete writes
   at transaction end.  Unfortuately, selects also wait which really
   isn't nessessary.  I hoped this would speed
   simultaneous connections.  I created this as both
   a bound thread with its own lwp and a threads manager
   managed thread.  I eventually settled on a threads manager
   managed thread, thinking that I wanted to set the priority
   of this thread low and commit as many transactions as
   possible simulateously.  Maybe, I should rethink this.
   
   As for client threads, that is managed by TAO and I
   haven't really played with that configuration.
   
   Myron Scott
   mkscott@sacadia.com
   
   
   

4. Re: Using Threads?

   Alfred Perlstein <bright@wintelcom.net> — 2001-01-02T17:24:25Z

   * Myron Scott <mscott@sacadia.com> [010102 08:47] wrote:
   > 
   > 
   > Alfred Perlstein wrote:
   > 
   > 
   > > 
   > > It's possible what you're seeing is the entire process
   > > wait for a disk IO to complete.
   > > 
   > > I'm wondering, how many lwps does your system use?  Are all
   > > the threads bound to a single lwp or do you let the threads
   > > manager handle this all for you?
   > > 
   > 
   > Yeah, I looked at this.  I have one thread per process that
   > does all flushing of buffer pages at transaction commit.
   > The client threads register buffer writes with this thread and
   > wait for this thread to complete writes
   > at transaction end.  Unfortuately, selects also wait which really
   > isn't nessessary.  I hoped this would speed
   > simultaneous connections.  I created this as both
   > a bound thread with its own lwp and a threads manager
   > managed thread.  I eventually settled on a threads manager
   > managed thread, thinking that I wanted to set the priority
   > of this thread low and commit as many transactions as
   > possible simulateously.  Maybe, I should rethink this.
   > 
   > As for client threads, that is managed by TAO and I
   > haven't really played with that configuration.
   
   I think you're saying you have an lwp in what you used to
   be per-process context?  Meaning if you have 4 connections,
   you should have 4 lwps doing disk IO?
   
   If that's true, most likely your problems may be elsewhere,
   but since you seem to be overcaching your writes you may
   be pessimizing the writes.  I wouldn't try to buffer it all,
   instead I would hold off until a write is done that isn't
   contiguous to the last write position and then being a flush
   operation.
   
   -- 
   -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
   "I have the heart of a child; I keep it in a jar on my desk."