Horton

Re: Compressed TOAST Slicing

Tom Lane <tgl@sss.pgh.pa.us>

View thread
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andres Freund <andres@anarazel.de>
Cc: Paul Ramsey <pramsey@cleverelephant.ca>, Andrey Borodin <x4mmm@yandex-team.ru>, Владимир Лесков <vladimirlesk@yandex-team.ru>, Stephen Frost <sfrost@snowman.net>, Daniel Verite <daniel@manitou-mail.org>, Simon Riggs <simon@2ndquadrant.com>, rafia.sabih@enterprisedb.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2019-04-09T17:30:22Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add support for partial TOAST decompression

  2. Remove remaining hard-wired OID references in the initial catalog data.

  3. Rephrase references to "time qualification".

Andres Freund <andres@anarazel.de> writes:
> On 2019-04-09 10:12:56 -0700, Paul Ramsey wrote:
>> Wow, well beyond slicing, just being able to decompress 25% faster is a win for pretty much any TOAST use case. I guess the $100 question is: portability? The whole reason for the old-skool code that’s there now was concerns about memcpy’ing overlapping addresses and Bad Things happening.

> Just use memmove? It's usually as fast these days.

If I recall what this is trying to do, memmove will give the wrong
result.  We want the expansion to replicate the same data multiple
times, which in normal use of memcpy/memmove would be thought to be
the Wrong Thing.

The proposal is kind of cute, but I'll bet it's a net loss for
small copy lengths --- likely we'd want some cutoff below which
we do it with the dumb byte-at-a-time loop.

			regards, tom lane