Thread

1. Re: tweaking MemSet() performance - 7.4.5

   Marc Colosimo <mcolosimo@mitre.org> — 2004-09-20T22:20:12Z

   >Marc Colosimo wrote:
   >
   >> Oops, I used the same setting as in the old hacking message (-O2, gcc 
   >> 3.3). If I understand what you are saying, then it turns out yes, PG's 
   >> MemSet is faster for smaller blocksizes (see below, between 32 and 
   >> 64). I just replaced the whole MemSet with memset and it is not very 
   >> low when I profile.
   >
   >Could you check what the OS-X memset function does internally?
   >One trick to speed up memset it to bypass the cache and bulk-write 
   >directly from write buffers to main memory. i386 cpus support that and 
   >in microbenchmarks it's 3 times faster (or something like that). 
   >Unfortunately it's a loss in real-world tests: Typically a structure is 
   >initialized with memset and then immediately accessed. If the memset 
   >bypasses the cache then the following access will cause a cache line 
   >miss, which can be so slow that using the faster memset can result in a 
   >net performance loss.
   >
   
   Could you suggest some structs to test? If I get your meaning, I would make a loop that sets then reads from the structure. 
   
   >> I could squeeze more out of it if I spent more time trying to 
   >> understand it (change MEMSET_LOOP_LIMIT to 32 and then add memset 
   >> after that?). I'm now working one understanding  Spin Locks and 
   >> friends. Putting in a sync call (in s_lock.h) is really a time killer 
   >> and bad for performance (it takes up 35 cycles).
   >>
   >That's the price you pay for weakly ordered memory access.
   >Linux on ppc uses eieio, on ppc64 lwsync is used. Could you check if 
   >they are faster?
   >
   
   I found the reason why "sync" was put in <http://archives.postgresql.org/pgsql-bugs/2002-09/msg00239.php>, but it is odd why it works. Why syncing one processor prevents the other from doing something is interesting. What type of shared memory is being used on OS X? I'm confused about the two types of semaphores, sysV or POSIX. <http://archives.postgresql.org/pgsql-patches/2001-01/msg00052.php>It seems the POSIX is the way to go on OS X.
   
   Marc
   
   
   
   
   

2. Re: tweaking MemSet() performance - 7.4.5

   Manfred Spraul <manfred@colorfullife.com> — 2004-09-25T21:23:13Z

   mcolosimo@mitre.org wrote:
   
   >>If the memset 
   >>bypasses the cache then the following access will cause a cache line 
   >>miss, which can be so slow that using the faster memset can result in a 
   >>net performance loss.
   >>
   >>    
   >>
   >
   >Could you suggest some structs to test? If I get your meaning, I would make a loop that sets then reads from the structure. 
   >
   >  
   >
   Read the sources and the cpu specs. Benchmarking such problems is 
   virtually impossible.
   I don't have OS-X, thus I checked the Linux-kernel sources: It seems 
   that the power architecture doesn't have the same problem as x86.
   There is a special clear cacheline instruction for large memsets and the 
   rest is done through carefully optimized store byte/halfword/word/double 
   word sequences.
   
   Thus I'd check what happens if you memset not perfectly aligned buffers. 
   That's another point where over-optimized functions sometimes break 
   down. If there is no slowdown, then I'd replace the postgres function 
   with the OS provided function.
   
   I'd add some __builtin_constant_p() optimizations, but I guess Tom won't 
   like gcc hacks ;-)
   --
       Manfred
   
   

3. Re: tweaking MemSet() performance - 7.4.5

   Karel Zak <zakkr@zf.jcu.cz> — 2004-09-29T08:46:18Z

   On Sat, 2004-09-25 at 23:23 +0200, Manfred Spraul wrote:
   > mcolosimo@mitre.org wrote:
   > 
   > >>If the memset 
   > >>bypasses the cache then the following access will cause a cache line 
   > >>miss, which can be so slow that using the faster memset can result in a 
   > >>net performance loss.
   > >>
   > >>    
   > >>
   > >
   > >Could you suggest some structs to test? If I get your meaning, I would make a loop that sets then reads from the structure. 
   > >
   > >  
   > >
   > Read the sources and the cpu specs. Benchmarking such problems is 
   > virtually impossible.
   > I don't have OS-X, thus I checked the Linux-kernel sources: It seems 
   > that the power architecture doesn't have the same problem as x86.
   > There is a special clear cacheline instruction for large memsets and the 
   > rest is done through carefully optimized store byte/halfword/word/double 
   > word sequences.
   > 
   > Thus I'd check what happens if you memset not perfectly aligned buffers. 
   > That's another point where over-optimized functions sometimes break 
   > down. If there is no slowdown, then I'd replace the postgres function 
   > with the OS provided function.
   > 
   > I'd add some __builtin_constant_p() optimizations, but I guess Tom won't 
   > like gcc hacks ;-)
   
   I think it cannot be problem if you write it to some .h file (in port
   directory?) as macro with "#ifdef GCC". The other thing is real
   advantage of hacks like this in practical PG usage :-)
   
   	Karel
   
   -- 
   Karel Zak
   http://home.zf.jcu.cz/~zakkr
   
   
   

4. Re: tweaking MemSet() performance - 7.4.5

   Bruce Momjian <pgman@candle.pha.pa.us> — 2004-09-29T11:37:00Z

   Karel Zak wrote:
   > On Sat, 2004-09-25 at 23:23 +0200, Manfred Spraul wrote:
   > > mcolosimo@mitre.org wrote:
   > > 
   > > >>If the memset 
   > > >>bypasses the cache then the following access will cause a cache line 
   > > >>miss, which can be so slow that using the faster memset can result in a 
   > > >>net performance loss.
   > > >>
   > > >>    
   > > >>
   > > >
   > > >Could you suggest some structs to test? If I get your meaning, I would make a loop that sets then reads from the structure. 
   > > >
   > > >  
   > > >
   > > Read the sources and the cpu specs. Benchmarking such problems is 
   > > virtually impossible.
   > > I don't have OS-X, thus I checked the Linux-kernel sources: It seems 
   > > that the power architecture doesn't have the same problem as x86.
   > > There is a special clear cacheline instruction for large memsets and the 
   > > rest is done through carefully optimized store byte/halfword/word/double 
   > > word sequences.
   > > 
   > > Thus I'd check what happens if you memset not perfectly aligned buffers. 
   > > That's another point where over-optimized functions sometimes break 
   > > down. If there is no slowdown, then I'd replace the postgres function 
   > > with the OS provided function.
   > > 
   > > I'd add some __builtin_constant_p() optimizations, but I guess Tom won't 
   > > like gcc hacks ;-)
   > 
   > I think it cannot be problem if you write it to some .h file (in port
   > directory?) as macro with "#ifdef GCC". The other thing is real
   > advantage of hacks like this in practical PG usage :-)
   
   The reason MemSet is a win is not that the C code is great but because
   it eliminates a function call.
   
   -- 
     Bruce Momjian                        |  http://candle.pha.pa.us
     pgman@candle.pha.pa.us               |  (610) 359-1001
     +  If your life is a hard drive,     |  13 Roberts Road
     +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
   
   

5. Re: tweaking MemSet() performance - 7.4.5

   Marc Colosimo <mcolosimo@mitre.org> — 2004-09-29T13:38:39Z

   On Sep 29, 2004, at 7:37 AM, Bruce Momjian wrote:
   
   > Karel Zak wrote:
   >> On Sat, 2004-09-25 at 23:23 +0200, Manfred Spraul wrote:
   >>> mcolosimo@mitre.org wrote:
   >>>
   >>>>> If the memset
   >>>>> bypasses the cache then the following access will cause a cache 
   >>>>> line
   >>>>> miss, which can be so slow that using the faster memset can result 
   >>>>> in a
   >>>>> net performance loss.
   >>>>
   >>>> Could you suggest some structs to test? If I get your meaning, I 
   >>>> would make a loop that sets then reads from the structure.
   >>>>
   >>> Read the sources and the cpu specs. Benchmarking such problems is
   >>> virtually impossible.
   >>> I don't have OS-X, thus I checked the Linux-kernel sources: It seems
   >>> that the power architecture doesn't have the same problem as x86.
   >>> There is a special clear cacheline instruction for large memsets and 
   >>> the
   >>> rest is done through carefully optimized store 
   >>> byte/halfword/word/double
   >>> word sequences.
   >>>
   >>> Thus I'd check what happens if you memset not perfectly aligned 
   >>> buffers.
   >>> That's another point where over-optimized functions sometimes break
   >>> down. If there is no slowdown, then I'd replace the postgres function
   >>> with the OS provided function.
   >>>
   
   all memory (via malloc and friends) will be aligned on OS X, unless you 
   remove padding (which I don't think you do)
   
   >>> I'd add some __builtin_constant_p() optimizations, but I guess Tom 
   >>> won't
   >>> like gcc hacks ;-)
   >>
   >> I think it cannot be problem if you write it to some .h file (in port
   >> directory?) as macro with "#ifdef GCC". The other thing is real
   >> advantage of hacks like this in practical PG usage :-)
   >
   > The reason MemSet is a win is not that the C code is great but because
   > it eliminates a function call.
   >
   
   Using MemSet really did speed things up. I think the function overhead 
   is okay. As for real world usage, the function ExecMakeFunctionResult 
   dropped from the top of the list when profiling (now < 1% vs 16% 
   before)!  This was doing a big nasty delete (w/ cascading), insert in a 
   cursor.
   
   Here are results for a Mac G4 (single processor) OS 10.3, using -O2. 
   This time the mac memset wins all around. Someone posted that this 
   wasn't the case.
   
   PG MemSet:
   pgmemset_test 32
   0.670u 0.020s 0:00.70 98.5%     0+0k 0+0io 0pf+0w
   pgmemset_test 64
   1.060u 0.000s 0:01.05 100.9%    0+0k 0+0io 0pf+0w
   pgmemset_test 128
   1.750u 0.010s 0:01.76 100.0%    0+0k 0+0io 0pf+0w
   pgmemset_test 512
   6.010u 0.030s 0:06.04 100.0%    0+0k 0+0io 0pf+0w
   
   Mac memset:
   memset_test 32
   0.660u 0.020s 0:00.67 101.4%    0+0k 0+0io 0pf+0w
   memset_test 64
   0.720u 0.000s 0:00.72 100.0%    0+0k 0+0io 0pf+0w
   memset_test 128
   0.800u 0.010s 0:00.81 100.0%    0+0k 0+0io 0pf+0w
   memset_test 512
   1.470u 0.010s 0:01.48 100.0%    0+0k 0+0io 0pf+0w
   
   Now I check about setting a byte after I memset, and it does slow down 
   a tiny bit. But it is the same for both MemSet and memset for under 64.
   
   
   
   

6. Re: tweaking MemSet() performance - 7.4.5

   Neil Conway <neilc@samurai.com> — 2004-09-30T00:40:48Z

   On Wed, 2004-09-29 at 21:37, Bruce Momjian wrote:
   > The reason MemSet is a win is not that the C code is great but because
   > it eliminates a function call.
   
   A reasonable compiler ought to be able to implement memset() as a
   compiler intrinsic where it makes sense to do so. MSVC++ can certainly
   do this; per the GCC 3.4 docs, it seems GCC can/does as well:
   
   The ISO C90 functions abort, abs, acos, asin, atan2, atan, calloc, ceil,
   cosh, cos, exit, exp, fabs, floor, fmod, fprintf, fputs, frexp, fscanf,
   labs, ldexp, log10, log, malloc, memcmp, memcpy, memset, modf, pow,
   printf, putchar, puts, scanf, sinh, sin, snprintf, sprintf, sqrt,
   sscanf, strcat, strchr, strcmp, strcpy, strcspn, strlen, strncat,
   strncmp, strncpy, strpbrk, strrchr, strspn, strstr, tanh, tan, vfprintf,
   vprintf and vsprintf are all recognized as built-in functions unless
   -fno-builtin is specified (or -fno-builtin-function is specified for an
   individual function). All of these functions have corresponding versions
   prefixed with __builtin_.
   
   (http://gcc.gnu.org/onlinedocs/gcc-3.4.2/gcc/Other-Builtins.html#Other-Builtins)
   
   -Neil
   
   
   
   

7. Re: tweaking MemSet() performance - 7.4.5

   Bruce Momjian <pgman@candle.pha.pa.us> — 2004-09-30T01:47:06Z

   Neil Conway wrote:
   > On Wed, 2004-09-29 at 21:37, Bruce Momjian wrote:
   > > The reason MemSet is a win is not that the C code is great but because
   > > it eliminates a function call.
   > 
   > A reasonable compiler ought to be able to implement memset() as a
   > compiler intrinsic where it makes sense to do so. MSVC++ can certainly
   > do this; per the GCC 3.4 docs, it seems GCC can/does as well:
   > 
   > The ISO C90 functions abort, abs, acos, asin, atan2, atan, calloc, ceil,
   > cosh, cos, exit, exp, fabs, floor, fmod, fprintf, fputs, frexp, fscanf,
   > labs, ldexp, log10, log, malloc, memcmp, memcpy, memset, modf, pow,
   > printf, putchar, puts, scanf, sinh, sin, snprintf, sprintf, sqrt,
   > sscanf, strcat, strchr, strcmp, strcpy, strcspn, strlen, strncat,
   > strncmp, strncpy, strpbrk, strrchr, strspn, strstr, tanh, tan, vfprintf,
   > vprintf and vsprintf are all recognized as built-in functions unless
   > -fno-builtin is specified (or -fno-builtin-function is specified for an
   > individual function). All of these functions have corresponding versions
   > prefixed with __builtin_.
   > 
   > (http://gcc.gnu.org/onlinedocs/gcc-3.4.2/gcc/Other-Builtins.html#Other-Builtins)
   
   MemSet was written when gcc 2.X wasn't even stable yet.  Have you run
   any tests on 3.4 to see if MemSet is still a win with that compiler?
   
   -- 
     Bruce Momjian                        |  http://candle.pha.pa.us
     pgman@candle.pha.pa.us               |  (610) 359-1001
     +  If your life is a hard drive,     |  13 Roberts Road
     +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
   
   

8. Re: tweaking MemSet() performance - 7.4.5

   Peter Eisentraut <peter_e@gmx.net> — 2004-10-01T20:40:03Z

   Bruce Momjian wrote:
   > MemSet was written when gcc 2.X wasn't even stable yet.  Have you run
   > any tests on 3.4 to see if MemSet is still a win with that compiler?
   
   I've done a test years ago that showed that memset is usually at least 
   as good as MemSet:
   
   http://archives.postgresql.org/pgsql-patches/2002-10/msg00085.php
   
   -- 
   Peter Eisentraut
   http://developer.postgresql.org/~petere/