Horton

Re: Compressed TOAST Slicing

Andrey Borodin <x4mmm@yandex-team.ru>

View thread
From: Andrey Borodin <x4mmm@yandex-team.ru>
To: Paul Ramsey <pramsey@cleverelephant.ca>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>, Michael Paquier <michael@paquier.xyz>, Andres Freund <andres@anarazel.de>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Regina Obe <r@pcorp.us>
Date: 2019-03-13T16:32:40Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add support for partial TOAST decompression

  2. Remove remaining hard-wired OID references in the initial catalog data.

  3. Rephrase references to "time qualification".


> 13 марта 2019 г., в 21:05, Paul Ramsey <pramsey@cleverelephant.ca> написал(а):
> 
> Here is a new (final?) patch ...
> 
> <compressed-datum-slicing-20190313a.patch>

This check

@@ -744,6 +748,8 @@ pglz_decompress(const char *source, int32 slen, char *dest,
 				{
 					*dp = dp[-off];
 					dp++;
+					if (dp >= destend)	/* check for buffer overrun */
+						break;		/* do not clobber memory */
 				}

is still done for every byte. You can precompute maximum allowed length before that cycle. Here's diff

diff --git a/src/common/pg_lzcompress.c b/src/common/pg_lzcompress.c
index 6b48892a8f..05b2b3d5d1 100644
--- a/src/common/pg_lzcompress.c
+++ b/src/common/pg_lzcompress.c
@@ -744,12 +744,11 @@ pglz_decompress_checked(const char *source, int32 slen, char *dest,
                                 * memcpy() here, because the copied areas could overlap
                                 * extremely!
                                 */
+                               len = Min(len, destend - dp);
                                while (len--)
                                {
                                        *dp = dp[-off];
                                        dp++;
-                                       if (dp >= destend)      /* check for buffer overrun */
-                                               break;          /* do not clobber memory */
                                }
                        }
                        else


Best regards, Andrey Borodin.