Re: Different compression methods for FPI

Justin Pryzby <pryzby@telsasoft.com>

From: Justin Pryzby <pryzby@telsasoft.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Andrey Borodin <x4mmm@yandex-team.ru>, pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@gmail.com>
Date: 2021-05-25T04:44:45Z
Lists: pgsql-hackers

Attachments

On Wed, May 19, 2021 at 06:31:15PM +0900, Michael Paquier wrote:
> I still don't understand why XID consistency has anything to do with
> the compression of FPIs.  There is nothing preventing the testing of
> compression of FPIs, and plese note this argument:
> https://www.postgresql.org/message-id/BEF3B1E0-0B31-4F05-8E0A-F681CB918626@yandex-team.ru
> 
> For example, I can just revert from my tree 0002 and 0003, and still
> perform tests of the various compression methods.  I do agree that we
> are going to need to do something about this problem, but let's drop
> this stuff from the set of patches of this thread and just discuss
> them where they are needed.

They are needed here - that they're included is deliberate.  Revert this and
then the tests fail.  "Make sure published XIDs are persistent"

time make -C src/test/recovery check
#   Failed test 'new xid after restart is greater'

> And you have not replaced BKPIMAGE_IS_COMPRESSED by a PGLZ-equivalent,
> so your patch set is eating more bits for BKPIMAGE_* than it needs

The goal is to support 2+ "methods" (including "none"), which takes 4 bits, so
may as well support 3 methods.

- uncompressed
- pglz
- lz4
- zlib or zstd or ??

This version:
0) repurposes the pre-existing GUC as an enum;
1) saves a bit (until zstd is included);
2) shows the compression in pg_waldump;

To support different compression levels, I think I'd change from an enum to
string and an assign hook, which sets a pair of ints.

-- 
Justin

Commits

  1. Add more error context to RestoreBlockImage() and consume it

  2. Add support for LZ4 with compression of full-page writes in WAL

  3. Extended statistics on expressions

  4. Be clear about whether a recovery pause has taken effect.

  5. Add GUC to enable compression of full page images stored in WAL.