Thread

1. TOAST and TEXT

   Chris Bitmead <chris@bitmead.com> — 2001-10-10T01:33:04Z

   Hi,
   
   Now that postgresql doesn't have field size limits, it seems to
   me they should be good for storing large blobs, even if it means
   having to uuencode them to be non-binary or whatever. I don't
   like the old large object implementation, I need to store very large
   numbers of objects and unless this implementation has changed
   in recent times it won't cut it.
   
   So my question is, I assume TEXT is the best data type to store
   large things in, what precisely is the range of characters that
   I can store in TEXT? Is it only characters ascii <= 127, or is
   it only printable characters, or everything except '\0' or what?
   
   
   

2. Re: TOAST and TEXT

   Rod Taylor <rbt@barchord.com> — 2001-10-10T01:45:23Z

   It should be noted that there is still a limit of about 1GB if I
   remember correctly.
   --
   Rod Taylor
   
   There are always four sides to every story: your side, their side, the
   truth, and what really happened.
   
   ----- Original Message -----
   From: "Chris Bitmead" <chris@bitmead.com>
   To: <pgsql-hackers@postgresql.org>
   Sent: Tuesday, October 09, 2001 9:33 PM
   Subject: [HACKERS] TOAST and TEXT
   
   
   > Hi,
   >
   > Now that postgresql doesn't have field size limits, it seems to
   > me they should be good for storing large blobs, even if it means
   > having to uuencode them to be non-binary or whatever. I don't
   > like the old large object implementation, I need to store very large
   > numbers of objects and unless this implementation has changed
   > in recent times it won't cut it.
   >
   > So my question is, I assume TEXT is the best data type to store
   > large things in, what precisely is the range of characters that
   > I can store in TEXT? Is it only characters ascii <= 127, or is
   > it only printable characters, or everything except '\0' or what?
   >
   >
   > ---------------------------(end of
   broadcast)---------------------------
   > TIP 6: Have you searched our list archives?
   >
   > http://archives.postgresql.org
   >
   
   
   

3. Re: TOAST and TEXT

   Marko Kreen <marko@l-t.ee> — 2001-10-10T02:48:42Z

   On Wed, Oct 10, 2001 at 11:33:04AM +1000, Chris Bitmead wrote:
   > So my question is, I assume TEXT is the best data type to store
   > large things in, what precisely is the range of characters that
   > I can store in TEXT? Is it only characters ascii <= 127, or is
   > it only printable characters, or everything except '\0' or what?
   
   text accepts everything except \0, and also various funtions
   take locale/charset info into account.
   
   Use bytea, its for 0-255, binary data.  When your client
   library does not support it, then base64 it in client side
   and later decode() into place.
   
   -- 
   marko
   
   
   

4. Re: TOAST and TEXT

   Jan Wieck <janwieck@yahoo.com> — 2001-10-10T12:52:54Z

   Rod Taylor wrote:
   > It should be noted that there is still a limit of about 1GB if I
   > remember correctly.
   
       You're right, there is still a practical limit on the size of
       a text field. And it's usually much lower than 1GB.
   
       The problem is that first, the (encoded) data has to  be  put
       completely  into  the  querystring, passed to the backend and
       buffered there entirely in memory. Then it get's parsed,  and
       the  data  copied  into  a  const  node.  After rewriting and
       planning, a  heap  tuple  is  build,  containing  the  third,
       eventually fourth in memory copy of the data. After that, the
       toaster kicks in, allocates another chunk of that size to try
       to compress the data and finally slices it up for storage.
   
       So the limit depends on how much swapspace you have and where
       the per process virtual memory limit of your OS is.
   
       In practice, sizes of up to 10 MB are no problem. So  storing
       typical MP3s works.
   
   
   Jan
   
   --
   
   #======================================================================#
   # It's easier to get forgiveness for being wrong than for being right. #
   # Let's break this rule - forgive me.                                  #
   #================================================== JanWieck@Yahoo.com #
   
   
   
   _________________________________________________________
   Do You Yahoo!?
   Get your free @yahoo.com address at http://mail.yahoo.com
   
   
   

5. Re: TOAST and TEXT

   Tom Lane <tgl@sss.pgh.pa.us> — 2001-10-11T04:32:47Z

   Chris Bitmead <chris@bitmead.com> writes:
   > ... I don't
   > like the old large object implementation, I need to store very large
   > numbers of objects and unless this implementation has changed
   > in recent times it won't cut it.
   
   Have you looked at 7.1?  AFAIK it has no particular problem with
   lots of LOs.
   
   Which is not to discourage you from going over to bytea fields instead,
   if that model happens to be more convenient for your application.
   But your premise above seems false.
   
   			regards, tom lane