Horton

Re: [HACKERS] Custom compression methods

Dilip Kumar <dilipbalaut@gmail.com>

View thread View on postgr.es
From: Dilip Kumar <dilipbalaut@gmail.com>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Robert Haas <robertmhaas@gmail.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Alexander Korotkov <a.korotkov@postgrespro.ru>, David Steele <david@pgmasters.net>, Ildus Kurbangaliev <i.kurbangaliev@gmail.com>, Dmitry Dolgov <9erthalion6@gmail.com>, PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>
Date: 2020-12-01T09:45:04Z
Lists: pgsql-hackers 
On Wed, Nov 25, 2020 at 12:50 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2020-Nov-24, Tom Lane wrote:
>
> > Robert Haas <robertmhaas@gmail.com> writes:
>
> > > Oh, I thought it had been suggested in previous discussions that these
> > > should be treated as access methods rather than inventing a whole new
> > > concept just for this, and it seemed like a good idea to me. I guess I
> > > missed the fact that the patch wasn't doing it that way. Hmm.
> >
> > FWIW, I kind of agree with Robert's take on this.  Heap and index AMs
> > are pretty fundamentally different animals, yet we don't have a problem
> > sticking them in the same catalog.  I think anything that is related to
> > storage access could reasonably go into that catalog, rather than
> > inventing a new one.
>
> Right -- Something like amname=lz4, amhandler=lz4handler, amtype=c.
> The core code must of course know how to instantiate an AM of type
> 'c' and what to use it for.
>
> https://postgr.es/m/20171213151818.75a20259@postgrespro.ru

I have changed this, I agree that using the access method for creating
compression has simplified the code.  I will share the updated patch
set after fixing other review comments by Robert.

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Commits

  1. docs: Update TOAST storage docs for configurable compression.

  2. Further tweaking of pg_dump's handling of default_toast_compression.

  3. Fix interaction of TOAST compression with expression indexes.

  4. Tidy up more loose ends related to configurable TOAST compression.

  5. Short-circuit slice requests that are for more than the object's size.

  6. Mostly-cosmetic adjustments of TOAST-related macros.

  7. Remove useless configure probe for <lz4/lz4.h>.

  8. Error on invalid TOAST compression in CREATE or ALTER TABLE.

  9. docs: Fix omissions related to configurable TOAST compression.

  10. More code cleanup for configurable TOAST compression.

  11. Bring configure support for LZ4 up to snuff.

  12. Make compression.sql regression test independent of default.

  13. Use valid compression method in brin_form_tuple

  14. Fix up pg_dump's handling of per-attribute compression options.

  15. Allow configurable LZ4 TOAST compression.

  16. Fix inconsistencies in the code

  17. Mostly-cosmetic improvements in memory chunk header alignment coding.

  18. Allow numeric to use a more compact, 2-byte header in many cases.