Thread
-
Re: [HACKERS] tables >2GB
Bruce Momjian <maillist@candle.pha.pa.us> — 1998-03-20T02:47:48Z
> > So, how about we fix the storage manager instead? > > I don't recall, what exactly breaks when going over 2 gig? I don't have the > disk space available, otherwise I'd debug this. I can still try if I knew > what the problem was...this code isn't all that complex. I agree. > OK...here is a patch that will cause the magnetic disk storage manager to > not try to split files in 2 gig chunks. It will just try to get another > block. > > If applied, everything is just as before. But if LET_OS_MANAGE_FILESIZE > is defined, the chaining disappears and the file just keeps on going, > and going, and going, til the OS barfs. > > Are there #defines in the system includes that could be used to determine > a max file size? If so, then I'd think that this would be something > to add to configure. If files over 2 gig are not allowed, then the old > code would compile. > > Anyway, if the patch looks ok to the powers-that-be or if there is some > thing else to be changed, let me know and I'll resubmit it to PATCHES. > > Compiled and regressed ok with and without LET_OS_MANAGE_FILESIZE, but > then again there aren't any regression tables over 2 gig. :) > Well, BSD/OS goes over 2gig, but the postgreSQL code uses lseek, which returns long, so even though I can handle larger files, the lseek() can't because long is 32-bits. Looks like only Alpha can handle those files based on our current code. Thanks to our success, this looks like something we will have to deal with. -- Bruce Momjian | 830 Blythe Avenue maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026 + If your life is a hard drive, | (610) 353-9879(w) + Christ can be your backup. | (610) 853-3000(h)
-
Re: [HACKERS] tables >2GB
Bruce Momjian <maillist@candle.pha.pa.us> — 1998-03-20T04:20:53Z
Applied. > I don't recall, what exactly breaks when going over 2 gig? I don't have the > disk space available, otherwise I'd debug this. I can still try if I knew > what the problem was...this code isn't all that complex. > > OK...here is a patch that will cause the magnetic disk storage manager to > not try to split files in 2 gig chunks. It will just try to get another > block. > > If applied, everything is just as before. But if LET_OS_MANAGE_FILESIZE > is defined, the chaining disappears and the file just keeps on going, > and going, and going, til the OS barfs. > > Are there #defines in the system includes that could be used to determine > a max file size? If so, then I'd think that this would be something > to add to configure. If files over 2 gig are not allowed, then the old > code would compile. > > Anyway, if the patch looks ok to the powers-that-be or if there is some -- Bruce Momjian | 830 Blythe Avenue maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026 + If your life is a hard drive, | (610) 353-9879(w) + Christ can be your backup. | (610) 853-3000(h)
-
Re: [HACKERS] tables >2GB
Tom Ivar Helbekkmo <tih@hamartun.priv.no> — 1998-03-20T16:40:01Z
* Bruce Momjian | | Well, BSD/OS goes over 2gig, but the postgreSQL code uses lseek, which | returns long, so even though I can handle larger files, the lseek() | can't because long is 32-bits. Are you sure? In NetBSD, lseek() is declared to return an off_t, which again is defined to be a 64bit quantity. I would assume that BSD/OS did it the same way -- in fact, I'd be surprised if not. -tih -- Popularity is the hallmark of mediocrity. --Niles Crane, "Frasier"
-
Re: [HACKERS] tables >2GB
Bruce Momjian <maillist@candle.pha.pa.us> — 1998-03-20T17:29:18Z
> > * Bruce Momjian > | > | Well, BSD/OS goes over 2gig, but the postgreSQL code uses lseek, which > | returns long, so even though I can handle larger files, the lseek() > | can't because long is 32-bits. > > Are you sure? In NetBSD, lseek() is declared to return an off_t, > which again is defined to be a 64bit quantity. I would assume that > BSD/OS did it the same way -- in fact, I'd be surprised if not. Oops, you are right: typedef quad_t off_t; I thought they added fgetpos() only for 64-bit quantities, and did not change the return value of lseek. However: sys/types.h:76: typedef quad_t off_t; /* file offset*/ so you are right, but our code: fd.c:110: long seekPos; fd.c:263: fileP->seekPos = (long) lseek(fileP->fd, 0L, SEEK_CUR); so it still will not work because the code is not defining seekPos as off_t. We need to get this code cleaned up/fixed. How could they make such a mistake and assume it is a long, unless this thing gets passed around in the backend, and they don't want to reference off_t all over the place? That code needs cleanup. -- Bruce Momjian | 830 Blythe Avenue maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026 + If your life is a hard drive, | (610) 353-9879(w) + Christ can be your backup. | (610) 853-3000(h)
-
Re: [HACKERS] tables >2GB
Tom Ivar Helbekkmo <tih@hamartun.priv.no> — 1998-03-21T12:52:29Z
* Bruce Momjian | | How could they make such a mistake and assume it is a long, [...] "All the world's a VAX!" Getting rid of (and tidying up after) explicit "long lseek();" declarations is a major part of porting old software to new UNIXes. -tih -- Popularity is the hallmark of mediocrity. --Niles Crane, "Frasier"
-
Re: [HACKERS] tables >2GB
Bruce Momjian <maillist@candle.pha.pa.us> — 1998-03-21T14:56:58Z
> > * Bruce Momjian > | > | How could they make such a mistake and assume it is a long, [...] > > "All the world's a VAX!" Getting rid of (and tidying up after) > explicit "long lseek();" declarations is a major part of porting > old software to new UNIXes. It's on our TODO list now. -- Bruce Momjian | 830 Blythe Avenue maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026 + If your life is a hard drive, | (610) 353-9879(w) + Christ can be your backup. | (610) 853-3000(h)