Horton

Different compression methods for FPI

x4mmm@yandex-team.ru

View thread View on postgr.es
From: Andrey Borodin <x4mmm@yandex-team.ru>
To: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2021-02-27T07:43:52Z
Lists: pgsql-hackers

Attachments

Hi!

There is a lot of different compression threads nearby. And that's great!
Every few bytes going to IO still deserve to be compressed.

Currently, we have a pglz compression for WAL full page images. As shown in [0] this leads to high CPU usage in pglz when wal_compression is on. Swapping pglz with lz4 increases pgbench tps by 21% on my laptop (if wal_compression is enabled).
So I think it worth to propose a patch to make wal_compression_method = {"pglz", "lz4", "zlib"}. Probably, "zstd" can be added to the list.

Even better option would be to teach WAL compression to compress everything, not only FPIs. But this is a big orthogonal chunk of work.

Attached is a draft taking CompressionId from "custom compression methods" patch and adding zlib to it.
I'm not sure where to add tests that check recovery with different methods. It seems to me that only TAP tests are suitable for this.

Thanks!

Best regards, Andrey Borodin.

[0] https://www.postgresql.org/message-id/323B1B01-DA42-419F-A99C-23E2C162D53B%40yandex-team.ru

Commits

  1. Add more error context to RestoreBlockImage() and consume it

  2. Add support for LZ4 with compression of full-page writes in WAL

  3. Extended statistics on expressions

  4. Be clear about whether a recovery pause has taken effect.

  5. Add GUC to enable compression of full page images stored in WAL.