Horton

Thread

  1. patch for memory overrun on Linux(i386)

    Maurice Gittens <mgittens@david.gits.nl> — 1998-03-21T18:44:12Z

    Hi,

It turns of that one of the bugs I detected with Electric Fence
is caused by an assumption that memory return by palloc and malloc 
is aligned on 8 bytes boundaries (double alignment). 
Adjusting this to correspond with the reality of the malloc
implementation on my pentium based linux box fixes the problem.

The following simple patch to include/utils/memutils.h
will fix the problem.

85c85
< #if ! defined(sco)
---
> #if ! defined(sco) && !defined(linux)

There is still another buffer overrun which I will try to find. 
The remaining one is triggered by creating a sequence.

I hope this patch will be applied if it makes sense to 
you guys.

With regards from Maurice.

  2. Re: [HACKERS] patch for memory overrun on Linux(i386)

    Marc G. Fournier <scrappy@hub.org> — 1998-03-21T23:55:39Z

    Some things just require confirmation...can someone else (Thomas?) running
Linux comment on this before I apply it?  I just find it kinda unexpected
to see something that doesn't work for SCO doesn't work for Linux too :)

On Sat, 21 Mar 1998, Maurice Gittens wrote:

> Hi,
> 
> It turns of that one of the bugs I detected with Electric Fence
> is caused by an assumption that memory return by palloc and malloc 
> is aligned on 8 bytes boundaries (double alignment). 
> Adjusting this to correspond with the reality of the malloc
> implementation on my pentium based linux box fixes the problem.
> 
> The following simple patch to include/utils/memutils.h
> will fix the problem.
> 
> 85c85
> < #if ! defined(sco)
> ---
> > #if ! defined(sco) && !defined(linux)
> 
> There is still another buffer overrun which I will try to find. 
> The remaining one is triggered by creating a sequence.
> 
> I hope this patch will be applied if it makes sense to 
> you guys.
> 
> With regards from Maurice.
> 
> 

Marc G. Fournier                                
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org

  3. Re: [HACKERS] patch for memory overrun on Linux(i386)

    ocie@paracel.com — 1998-03-22T00:37:34Z

    Maurice Gittens wrote:
> 
> Hi,
> 
> It turns of that one of the bugs I detected with Electric Fence
> is caused by an assumption that memory return by palloc and malloc 
> is aligned on 8 bytes boundaries (double alignment). 
> Adjusting this to correspond with the reality of the malloc
> implementation on my pentium based linux box fixes the problem.
> 
> The following simple patch to include/utils/memutils.h
> will fix the problem.
> 
> 85c85
> < #if ! defined(sco)
> ---
> > #if ! defined(sco) && !defined(linux)

I may be off on this one, but I thought that memory alignment was a
cpu, and not necessarily an OS issue.  I.E. Solaris x86 might show the
"misalignment", while Linux Alpha would not.

Ocie Mitchell

  4. Re: [HACKERS] patch for memory overrun on Linux(i386)

    Marc G. Fournier <scrappy@hub.org> — 1998-03-22T01:26:18Z

    On Sat, 21 Mar 1998 ocie@paracel.com wrote:

> Maurice Gittens wrote:
> > 
> > Hi,
> > 
> > It turns of that one of the bugs I detected with Electric Fence
> > is caused by an assumption that memory return by palloc and malloc 
> > is aligned on 8 bytes boundaries (double alignment). 
> > Adjusting this to correspond with the reality of the malloc
> > implementation on my pentium based linux box fixes the problem.
> > 
> > The following simple patch to include/utils/memutils.h
> > will fix the problem.
> > 
> > 85c85
> > < #if ! defined(sco)
> > ---
> > > #if ! defined(sco) && !defined(linux)
> 
> I may be off on this one, but I thought that memory alignment was a
> cpu, and not necessarily an OS issue.  I.E. Solaris x86 might show the
> "misalignment", while Linux Alpha would not.

	Ummm...you are comparing both two different CPU and two different
OSs here...more appropriately to your argument would be Linux/x86 vs
Linux/Alpha, wouldn't it?  If it was a CPU issue...?

Marc G. Fournier                                
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org

  5. Re: [HACKERS] patch for memory overrun on Linux(i386)

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-03-22T03:58:03Z

    > 
> 
> Some things just require confirmation...can someone else (Thomas?) running
> Linux comment on this before I apply it?  I just find it kinda unexpected
> to see something that doesn't work for SCO doesn't work for Linux too :)

On BSDI, malloc man pages says:

     The allocated space is suitably aligned (after possible pointer coercion)
     for storage of any type of object.

I don't believe this is a standard, and I think Linux may not follow it.
I think I heard somewhere they don't.  We certainly need to have this
checked before use in a patch.


-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

  6. Re: [HACKERS] patch for memory overrun on Linux(i386)

    Marc Howard Zuckman <marc@fallon.classyad.com> — 1998-03-22T04:57:51Z

    On Sat, 21 Mar 1998, Bruce Momjian wrote:

> > 
> > 
> > Some things just require confirmation...can someone else (Thomas?) running
> > Linux comment on this before I apply it?  I just find it kinda unexpected
> > to see something that doesn't work for SCO doesn't work for Linux too :)
> 
> On BSDI, malloc man pages says:
> 
>      The allocated space is suitably aligned (after possible pointer coercion)
>      for storage of any type of object.
> 
> I don't believe this is a standard, and I think Linux may not follow it.
> I think I heard somewhere they don't.  We certainly need to have this
> checked before use in a patch.
> 
> 
> -- 
> Bruce Momjian                          |  830 Blythe Avenue
> maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
>   +  If your life is a hard drive,     |  (610) 353-9879(w)
>   +  Christ can be your backup.        |  (610) 853-3000(h)
> 

>From the linux man page:
 For calloc() and malloc(), the value returned is a pointer
       to the allocated memory, which is suitably aligned for any
       kind of variable, or NULL if the request fails.

       realloc() returns a pointer to the newly allocated memory,
       which is suitably aligned for any kind of variable and may
       be  different from ptr, or NULL if the request fails or if
       size was equal to 0.   If  realloc()  fails  the  original
       block is left untouched - it is not freed or moved.




Marc Zuckman
marc@fallon.classyad.com

_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
_     Visit The Home and Condo MarketPlace		      _
_          http://www.ClassyAd.com			      _
_							      _
_  FREE basic property listings/advertisements and searches.  _
_							      _
_  Try our premium, yet inexpensive services for a real	      _
_   selling or buying edge!				      _
_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

  7. Re: [HACKERS] patch for memory overrun on Linux(i386)

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-03-22T05:21:22Z

    > >From the linux man page:
>  For calloc() and malloc(), the value returned is a pointer
>        to the allocated memory, which is suitably aligned for any
>        kind of variable, or NULL if the request fails.

OK, then why is electric fence telling him to make DOUBLEALIGN match
INTALIGN?  You would think that if this change made sense, we would be
having some Linux problems, but we don't.

---------------------------------------------------------------------------


#if ! defined(sco)
#define DOUBLEALIGN(LEN)\
        (((long)(LEN) + (sizeof (double) - 1)) & ~(sizeof (double) -1))

#define MAXALIGN(LEN)\
        (((long)(LEN) + (sizeof (double) - 1)) & ~(sizeof (double) -1))
#else
#define DOUBLEALIGN(LEN) INTALIGN(LEN)
#define MAXALIGN(LEN)    INTALIGN(LEN)
#endif



-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

  8. Re: [HACKERS] patch for memory overrun on Linux(i386)

    David Gould <dg@illustra.com> — 1998-03-22T08:39:42Z

    > > Some things just require confirmation...can someone else (Thomas?) running
> > Linux comment on this before I apply it?  I just find it kinda unexpected
> > to see something that doesn't work for SCO doesn't work for Linux too :)
> 
> On BSDI, malloc man pages says:
> 
>      The allocated space is suitably aligned (after possible pointer coercion)
>      for storage of any type of object.
> 
> I don't believe this is a standard, and I think Linux may not follow it.
> I think I heard somewhere they don't.  We certainly need to have this
> checked before use in a patch.

The alignment of a pointer returned from malloc() is platform specific. The
BSDI man page above gets the sense of it right, and as far as I am aware
all mallocs conform to this restriction. In particular x86 Linux malloc
behaves exactly that way.

Of course on x86 there is no special requirement any type to be aligned at all.
Double do not need to be 8 byte aligned. Ints do not need to be on an even
address etc. So malloc can return a pointer to any address it wants on an x86
and still conform to the restriction.

On a Sparc or an Alpha, objects must to be aligned to the sizeof the type. 
So shorts are 2 byte aligned, ints 4 byte, and doubles 8 byte. Since malloc
has no idea what you are allocating it has to return an 8 byte aligned pointer.

But all this is a bit of a red herring. The original bug if I recall was
caused by the caller assuming that the return from palloc was 8 byte aligned.
Since the caller "knew" that the pointer was 8 byte aligned it then went on
to do some "fancy" pointer arithmetic to arrive at an offset. This is what
is broken, not the Linux malloc().

That said, I have a plan in mind to get a nice performance boost for palloc().
As a side effect, I will guarantee 8 byte alignment of pointers returned
from palloc().

-dg

David Gould            dg@illustra.com           510.628.3783 or 510.305.9468 
Informix Software  (No, really)         300 Lakeside Drive  Oakland, CA 94612
 - I realize now that irony has no place in business communications.

  9. Re: [HACKERS] patch for memory overrun on Linux(i386)

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-03-22T16:55:58Z

    > That said, I have a plan in mind to get a nice performance boost for palloc().
> As a side effect, I will guarantee 8 byte alignment of pointers returned
> from palloc().

Oooh, sounds nice.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

  10. Re: [HACKERS] patch for memory overrun on Linux(i386)

    Massimo Dal Zotto <dz@cs.unitn.it> — 1998-03-26T15:16:55Z

    > 
> > >From the linux man page:
> >  For calloc() and malloc(), the value returned is a pointer
> >        to the allocated memory, which is suitably aligned for any
> >        kind of variable, or NULL if the request fails.
> 
> OK, then why is electric fence telling him to make DOUBLEALIGN match
> INTALIGN?  You would think that if this change made sense, we would be
> having some Linux problems, but we don't.
> 
> ---------------------------------------------------------------------------
> 
> 
> #if ! defined(sco)
> #define DOUBLEALIGN(LEN)\
>         (((long)(LEN) + (sizeof (double) - 1)) & ~(sizeof (double) -1))
> 
> #define MAXALIGN(LEN)\
>         (((long)(LEN) + (sizeof (double) - 1)) & ~(sizeof (double) -1))
> #else
> #define DOUBLEALIGN(LEN) INTALIGN(LEN)
> #define MAXALIGN(LEN)    INTALIGN(LEN)
> #endif
> 

Hi Hackers,

it seems that Linux malloc, or better libc-5.4.23 malloc, is doing what is
declared in the man page, i.e. returning "memory which is suitably aligned
for any kind of variable". I have some malloc traces and all malloc results
are aligned to double size (8 bytes): 

op	caller		ptr		size
--------------------------------------------
malloc	400c32b2	080a06c8	25
malloc	400c32b2	080a06e8	13
malloc	400c32b2	080a0700	26
malloc	400c32b2	080a0720	11
malloc	400c32b2	080a0730	50
malloc	400c32b2	080a0768	14
malloc	400c32b2	080a0780	17
malloc	400c32b2	080a0798	24
malloc	400c32b2	080a07b8	27
malloc	400c32b2	080a07d8	817
malloc	400c32b2	080a0b10	11
malloc	400c32b2	080a0b20	14
malloc	400c32b2	080a0b38	164
malloc	400c32b2	080a0be0	11
malloc	400c32b2	080a0bf0	14
malloc	400c32b2	080a0c08	25
malloc	400c32b2	080a0c28	16
malloc	400c32b2	080a0c40	35
malloc	400c32b2	080a0c68	9
malloc	400c32b2	080a0c78	7

So I don't see where is the problem.

Massimo Dal Zotto

+----------------------------------------------------------------------+
|  Massimo Dal Zotto                e-mail:  dz@cs.unitn.it            |
|  Via Marconi, 141                 phone:  ++39-461-534251            |
|  38057 Pergine Valsugana (TN)     www:  http://www.cs.unitn.it/~dz/  |
|  Italy                            pgp:  finger dz@tango.cs.unitn.it  |
+----------------------------------------------------------------------+