Re: Different compression methods for FPI

x4mmm@yandex-team.ru

From: Andrey Borodin <x4mmm@yandex-team.ru>
To: Justin Pryzby <pryzby@telsasoft.com>
Cc: Michael Paquier <michael@paquier.xyz>, pgsql-hackers@postgresql.org
Date: 2021-03-06T07:29:14Z
Lists: pgsql-hackers

> 1 марта 2021 г., в 10:03, Justin Pryzby <pryzby@telsasoft.com> написал(а):

Justin, Michael, thanks for comments!

As far as I understood TODO list for the patch looks as follows:

1. Reuse compression API of other patches. But which one? Or should I invent new one? "compressamapi.h" from "custom compression methods" bothers only with varlena <-> varlena conversions, and only for lz4. And it is "access method" after all, residing in backend...
ZStream looks good, but it lacks OID identification for compression methods and lz4.
2. Store OID in FPIs instead of 1-byte CompressionId. Make sure frontend is able to recognize builtin compression OIDs.
3. Complain if wal_compression_method is set to lib which is not compiled in.
4. Add tests for different WAL compression methods similar to src/test/ssl

Did I miss something?
I would appreciate a help with item 1, I do not know how to choose starting point.

> wal_compression_method (why not just using wal_compression?)

I hope one day we will compress all WAL, not just FPIs. Advanced archive management tools already do so, why not compress it in walwriter?
When this will be implemented, we could have wal_compression = {off, fpi, all}.

Best regards, Andrey Borodin.


Commits

  1. Add more error context to RestoreBlockImage() and consume it

  2. Add support for LZ4 with compression of full-page writes in WAL

  3. Extended statistics on expressions

  4. Be clear about whether a recovery pause has taken effect.

  5. Add GUC to enable compression of full page images stored in WAL.