Horton

Re: [HACKERS] Custom compression methods

Dilip Kumar <dilipbalaut@gmail.com>

View thread View on postgr.es
From: Dilip Kumar <dilipbalaut@gmail.com>
To: Justin Pryzby <pryzby@telsasoft.com>
Cc: Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Alexander Korotkov <a.korotkov@postgrespro.ru>, David Steele <david@pgmasters.net>, Ildus Kurbangaliev <i.kurbangaliev@gmail.com>, Dmitry Dolgov <9erthalion6@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2021-03-16T05:59:18Z
Lists: pgsql-hackers 
On Tue, Mar 16, 2021 at 11:18 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
>
> I'm a minor contributor now to a couple bits of this patch set, but I can
> answer a couple of these points.
>
> On Mon, Mar 15, 2021 at 03:58:35PM -0700, Andres Freund wrote:
> > Comments about 0003:
> > - why is HIDE_TOAST_COMPRESSION useful? Doesn't quite seem to be
> >   comparable to HIDE_TABLEAM?
>
> That was my idea and implementation.
> It's because until 3 weeks ago, the patchset supported a "plugable compression
> API" like CREATE ACCESS METHOD, a suggestion from Alvaro to avoid making a new
> table and everything involved just for a few rows).  Now, the patch is limited
> to lz4, and the "pluggable compression APIs" isn't included in the latest
> patchsets.

Yeah, but now also it makes sense to hide the compression method to
avoid unrelated regression changes.  But I am okay if we think we want
to drop this?

> > Comments about 0005:
> > - I'm personally not really convinced tracking the compression type in
> >   pg_attribute the way you do is really worth it (. Especially given
> >   that it's right now only about new rows anyway.  Seems like it'd be
> >   easier to just treat it as a default for new rows, and dispense with
> >   all the logic around mismatching compression types etc?
>
> I made the half-serious suggestion to make it a per-relation relopt.
> That would allow implementing pg_dump --no-toast-compression, to allow
> restoring a dump from a server with LZ4 tables to a server --without-lz4.
> Similar to --no-tablespaces.

I am not sure how good an idea it is to support table-level options.
The attribute level option makes sense to me in case we want to
support different compression methods for different data types.
Currently, we have only pglz and lz4 but if we are not planning for
custom compression in the future then we can support 2 more built-in
compression methods so I still feel having an attribute level option
makes more sense.

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Commits

  1. docs: Update TOAST storage docs for configurable compression.

  2. Further tweaking of pg_dump's handling of default_toast_compression.

  3. Fix interaction of TOAST compression with expression indexes.

  4. Tidy up more loose ends related to configurable TOAST compression.

  5. Short-circuit slice requests that are for more than the object's size.

  6. Mostly-cosmetic adjustments of TOAST-related macros.

  7. Remove useless configure probe for <lz4/lz4.h>.

  8. Error on invalid TOAST compression in CREATE or ALTER TABLE.

  9. docs: Fix omissions related to configurable TOAST compression.

  10. More code cleanup for configurable TOAST compression.

  11. Bring configure support for LZ4 up to snuff.

  12. Make compression.sql regression test independent of default.

  13. Use valid compression method in brin_form_tuple

  14. Fix up pg_dump's handling of per-attribute compression options.

  15. Allow configurable LZ4 TOAST compression.

  16. Fix inconsistencies in the code

  17. Mostly-cosmetic improvements in memory chunk header alignment coding.

  18. Allow numeric to use a more compact, 2-byte header in many cases.