Horton

Re: Compressed TOAST Slicing

Andres Freund <andres@anarazel.de>

View thread
From: Andres Freund <andres@anarazel.de>
To: Paul Ramsey <pramsey@cleverelephant.ca>
Cc: Andrey Borodin <x4mmm@yandex-team.ru>, Владимир Лесков <vladimirlesk@yandex-team.ru>, Stephen Frost <sfrost@snowman.net>, Daniel Verite <daniel@manitou-mail.org>, Simon Riggs <simon@2ndquadrant.com>, rafia.sabih@enterprisedb.com, Tom Lane <tgl@sss.pgh.pa.us>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2019-04-09T17:20:49Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add support for partial TOAST decompression

  2. Remove remaining hard-wired OID references in the initial catalog data.

  3. Rephrase references to "time qualification".

On 2019-04-09 10:12:56 -0700, Paul Ramsey wrote:
> 
> > On Apr 9, 2019, at 10:09 AM, Andrey Borodin <x4mmm@yandex-team.ru> wrote:
> > 
> > He advised me to use algorithm that splits copied regions into smaller non-overlapping subregions with exponentially increasing size.
> > 
> > while (off <= len)
> > {
> >    memcpy(dp, dp - off, off);
> >    len -= off;
> >    dp += off;
> >    off *= 2;
> > }
> > memcpy(dp, dp - off, len);
> > 
> > On original Paul's test without patch of this thread this optimization gave about x2.5 speedup.
> > I've composed more detailed tests[0] and tested against current master. Now it only gives 20%-25% of decompression speedup, but I think it is still useful.
> 
> Wow, well beyond slicing, just being able to decompress 25% faster is a win for pretty much any TOAST use case. I guess the $100 question is: portability? The whole reason for the old-skool code that’s there now was concerns about memcpy’ing overlapping addresses and Bad Things happening.

Just use memmove? It's usually as fast these days.