Horton

Re: Compressed TOAST Slicing

Andrey Borodin <x4mmm@yandex-team.ru>

View thread
From: Andrey Borodin <x4mmm@yandex-team.ru>
To: Paul Ramsey <pramsey@cleverelephant.ca>
Cc: Stephen Frost <sfrost@snowman.net>, Daniel Verite <daniel@manitou-mail.org>, Simon Riggs <simon@2ndquadrant.com>, rafia.sabih@enterprisedb.com, Tom Lane <tgl@sss.pgh.pa.us>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2019-03-12T05:22:45Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add support for partial TOAST decompression

  2. Remove remaining hard-wired OID references in the initial catalog data.

  3. Rephrase references to "time qualification".

Hi!

> 21 февр. 2019 г., в 23:50, Paul Ramsey <pramsey@cleverelephant.ca> написал(а):
> 
> Merci! Attached are updated patches.
> <compressed-datum-slicing-20190221a.patch><compressed-datum-slicing-left-20190221a.patch>

As noted before, patches are extremely useful.
So, I've looked into the code too.

I've got some questions about pglz_decompress() changes:

1.
+					if (dp >= destend)	/* check for buffer overrun */
+						break;		/* do not clobber memory */
This is done inside byte-loop. But can we just calculate len = min(len, destend - dp) beforehand?

2. Function argument is_slice is only preventing error.

+	 * If we are slicing, then we won't necessarily
+	 * be at the end of the source or dest buffers
+	 * when we hit a stop, so we don't test then.

But I do not get why we should get that error. If we have limited dest, why we should not fill it entirely?

3. And I'd use memmove despite the comment why we do not do that. It is SSE-optimized and cache-optimized nowadays.
But this in not point of this patch, so let's discard this point.

Best regards, Andrey Borodin.