Re: ZStandard (with dictionaries) compression support for TOAST compression

Aleksander Alekseev <aleksander@timescale.com>

From: Aleksander Alekseev <aleksander@timescale.com>

To: pgsql-hackers@postgresql.org

Cc: Robert Haas <robertmhaas@gmail.com>

Date: 2025-03-07T11:56:01Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Split regression tests for TOAST compression methods into two files
- 74a3fc36f314 19 (unreleased) landed
Refactor non-supported compression error message in toast_compression.c
- 1dbe6f76677c 19 (unreleased) landed

Hi Robert,

> I think that solving the problems around using a dictionary is going
> to be really hard. Can we see some evidence that the results will be
> worth it?

Compression dictionaries give a good compression ratio (~50%) and also
increase TPS a bit (5-10%) due to better buffer cache utilization. At
least according to synthetic and not trustworthy benchmarks I did some
years ago [1]. The result may be very dependent on the actual data of
course, not to mention particular implementation of the idea.

[1]: https://github.com/afiskon/zson/blob/master/docs/benchmark.md

-- 
Best regards,
Aleksander Alekseev