Re: [HACKERS] Custom compression methods
Dilip Kumar <dilipbalaut@gmail.com>
Attachments
- v35-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patch (text/x-patch) patch v35-0001
- v35-0002-Expand-the-external-data-before-forming-the-tupl.patch (text/x-patch) patch v35-0002
- v35-0004-Add-default_toast_compression-GUC.patch (text/x-patch) patch v35-0004
- v35-0005-Alter-table-set-compression.patch (text/x-patch) patch v35-0005
- v35-0003-Built-in-compression-method.patch (text/x-patch) patch v35-0003
- v35-0006-Disallow-compressed-data-inside-container-types.patch (text/x-patch) patch v35-0006
- v35-0007-default-to-with-lz4.patch (text/x-patch) patch v35-0007
On Fri, Mar 12, 2021 at 4:06 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
>
> I think these names need to be more specific.
>
> +typedef enum CompressionId
> +{
> + PGLZ_COMPRESSION_ID = 0,
> + LZ4_COMPRESSION_ID = 1
> +
>
> CompressionId, PGLZ_COMPRESSION_ID, LZ4_COMPRESSION_ID are also being used by
> Andrey's WAL compression patch. I suggested he use a prefix, but your patch is
> also of limited scope (TOAST).
In the attached patches I have changed this, other than this I have
run pg_indent on all the patches and now CTAS and Matview will compare
the compression method with the target attribute and it will
recompress if required. CLUSTER, VACUUM
FULL(reform_and_rewrite_tuple) and ATRewriteTable will also recompress
as per the target attribute. But the changes of
reform_and_rewrite_tuple and ATRewriteTable are in patch 0005, because
now whenever we are moving to target table we are always recompress so
there is no use-case to recompress in reform_and_rewrite_tuple and
ATRewriteTable unless we provide an option to alter the attribute
compression. Basically, we are allowed to change the attribute
compression without any rewrite but on the very next opportunity when
the table is getting rewritten the data will be recompressed based on
the current compression method. Also done some improvement and
comment changes in patch 0002 and patch 0006.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Commits
-
docs: Update TOAST storage docs for configurable compression.
- e8c435a824e1 14.0 landed
-
Further tweaking of pg_dump's handling of default_toast_compression.
- 54bb91c30e39 14.0 landed
-
Fix interaction of TOAST compression with expression indexes.
- 5db1fd7823a1 14.0 landed
-
Tidy up more loose ends related to configurable TOAST compression.
- e5595de03ec6 14.0 landed
-
Short-circuit slice requests that are for more than the object's size.
- 063dd37ebc76 14.0 landed
-
Mostly-cosmetic adjustments of TOAST-related macros.
- aeb1631ed207 14.0 landed
-
Remove useless configure probe for <lz4/lz4.h>.
- 2c75f8a612b2 14.0 landed
-
Error on invalid TOAST compression in CREATE or ALTER TABLE.
- a4d5284a10b5 14.0 landed
-
docs: Fix omissions related to configurable TOAST compression.
- 24f0e395ac58 14.0 landed
-
More code cleanup for configurable TOAST compression.
- 226e2be3876d 14.0 landed
-
Bring configure support for LZ4 up to snuff.
- 4d399a6fbeb7 14.0 landed
-
Make compression.sql regression test independent of default.
- fd1ac9a54896 14.0 landed
-
Use valid compression method in brin_form_tuple
- 882b2cdc08c4 14.0 landed
-
Fix up pg_dump's handling of per-attribute compression options.
- aa25d1089ac0 14.0 landed
-
Allow configurable LZ4 TOAST compression.
- bbe0a81db69b 14.0 landed
-
Fix inconsistencies in the code
- 6b8548964bcc 13.0 cited
-
Mostly-cosmetic improvements in memory chunk header alignment coding.
- f65d21b25808 11.0 cited
-
Allow numeric to use a more compact, 2-byte header in many cases.
- 145343534c15 9.1.0 cited