Re: Compressed TOAST Slicing

On Thu, Nov 1, 2018 at 2:29 PM Stephen Frost <sfrost@snowman.net> wrote:

> Greetings,
>
> * Paul Ramsey (pramsey@cleverelephant.ca) wrote:
> > The attached patch adds in a code path to do a partial decompression of
> the
> > TOAST entry, when the requested slice is at the start of the object.
>
> There two things that I wonder about in the patch- if it would be of any
> use to try and allocate on a need basis instead of just allocating the
> whole chunk up to the toast size,


I'm not sure what I was thinking when I rejected allocating the slice size
in favour of the whole uncompressed size... I cannot see why that wouldn't
work.


> and secondly, why we wouldn't consider
> handling a non-zero offset.  A non-zero offset would, of course, still
> require decompressing from the start and then just throwing away what we
> skip over, but we're going to be doing that anyway, aren't we?  Why not
> stop when we get to the end, at least, and save ourselves the trouble of
> decompressing the rest and then throwing it away.
>

I was worried about changing the pg_lz code too much because it scared me,
but debugging some stuff made me read it more closely so I fear it less
now, and doing interior slices seems not unreasonable, so I will give it a
try.

P