Re: ZStandard (with dictionaries) compression support for TOAST compression

Nikita Malakhov <hukutoc@gmail.com>

From: Nikita Malakhov <hukutoc@gmail.com>

To: Nikhil Kumar Veldanda <veldanda.nikhilkumar17@gmail.com>

Cc: Robert Haas <robertmhaas@gmail.com>, Michael Paquier <michael@paquier.xyz>, pgsql-hackers@postgresql.org

Date: 2025-04-29T10:36:51Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Split regression tests for TOAST compression methods into two files
- 74a3fc36f314 19 (unreleased) landed
Refactor non-supported compression error message in toast_compression.c
- 1dbe6f76677c 19 (unreleased) landed

Hi,

Nikhil, please consider existing discussions on using dictionaries
(mentioned above by Aleksander) and extending the TOAST pointer [1],
it seems you did not check them.

The same question Robert asked above - it's unclear why the header
wastes so much space. You mentioned metadata length - what metadata
do you mean there?

Also Robert pointed out very questionable approaches in your solution -
new dependencies crawling around user tables, new catalog table
with very unclear lifecycle (and, having new catalog table, immediately
having questions with pg_upgrade).
Currently I'm looking through the patch and could share my thoughts
later.

While reading this thread I've thought about storing a dictionary within
the table it is used for - IIUC on dictionary is used for just one
attribute,
so it does not make sense to make it global.

Also, I have a question regarding the Zstd implementation you propose -
does it provide a possibility for partial decompression?

>
>
Thanks!

[1]
https://www.postgresql.org/message-id/flat/CAN-LCVMq2X%3Dfhx7KLxfeDyb3P%2BBXuCkHC0g%3D9GF%2BJD4izfVa0Q%40mail.gmail.com

-- 
Regards,
Nikita Malakhov
Postgres Professional
The Russian Postgres Company
https://postgrespro.ru/