Re: update on TOAST status'

Tom Lane wrote:
> JanWieck@t-online.de (Jan Wieck) writes:
> >     ... So do we want to  have  indices  storing  plain
> >     values allways and limit them in the index-tuple size or not?
>
> I think not: it will be seen as a robustness failure, even (or
> especially) if it doesn't happen often.  I can see the bug reports now:
> "Hey!  I tried to insert a long value in my field, and it didn't work!
> I thought you'd fixed this bug?"
>
> You make good arguments that we shouldn't be too concerned about the
> speed of access to toasted index values, and I'm willing to accept
> that point of view (at least till we have hard evidence about it).
> But when I say "it should be bulletproof" I mean it should *work*,
> without imposing arbitrary limits on the user.  Arbitrary limits are
> exactly what we are trying to eliminate.

    After debugging something I thought was a bug in the toaster,
    I've found something really causing headaches.

    TOAST AS IS IS NOT CAPABLE OF HOLDING INDEXED VALUES!

    It appears that brtee indices (at least) can keep  references
    to  old  toast  values  that  survive  a  VACUUM! Seems these
    references live in nodes actually not  referring  to  a  heap
    tuple   any   more,   but   used  during  tree  traversal  in
    comparisions.  As if an index tuple delete from a  btree  not
    necessarily  causes  the  index  value  to disappear from the
    btree completely.  It'll never be returned by an index  scan,
    but the value is still there somewhere.

    Everything  is  OK  with this up to a VACUUM run. The toaster
    uses SnapShotAny to fetch toast values. So an external  value
    can  be  fetched by the toaster even if it is already deleted
    and committed. If  he  has  a  reference  somewhere,  he  has
    allways   a  share  or  higher  lock  on  the  main  relation
    preventing VACUUM to mangle up the toast  relation  (I  moved
    toast relation vacuuming into the lock time of the main table
    recently).

    But in the above case it is already vacuumed and not  present
    any  more. Now the btree traversal needs to compare against a
    value, long gone to the bit heaven, and that cannot work with
    the toast architecture.

    Seems  the  designs of btree and toast are colliding. As soon
    as "text" is toastable, this'll hurt - be warned.

    AFAICS, we need to detoast values for index  inserts  allways
    and  have  another toaster inside the index access methods at
    some day.  In the meantime we should decide  a  safe  maximum
    index tuple size and emit an explanative error message on the
    attempt to insert oversized index entries instead of possibly
    corrupting the index.

    Comment!


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #