Horton

Re: Different compression methods for FPI

Michael Paquier <michael@paquier.xyz>

View thread View on postgr.es
From: Michael Paquier <michael@paquier.xyz>
To: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: Andrey Borodin <x4mmm@yandex-team.ru>, Justin Pryzby <pryzby@telsasoft.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Thomas Munro <thomas.munro@gmail.com>, Peter Geoghegan <pg@bowt.ie>, Andres Freund <andres@anarazel.de>
Date: 2021-06-17T01:12:04Z
Lists: pgsql-hackers 
On Wed, Jun 16, 2021 at 11:49:51AM +0300, Heikki Linnakangas wrote:
> Hmm, do we currently compress each block in a WAL record separately, for
> records that contain multiple full-page images? That could make a big
> difference e.g. for GiST index build that WAL-logs 32 pages in each record.
> If it helps the compression, we should probably start WAL-logging b-tree
> index build in larger batches, too.

Each block is compressed alone, see XLogCompressBackupBlock() in
XLogRecordAssemble() where we loop through each block.  Compressing a
group of blocks would not be difficult (the refactoring may be
trickier than it looks) but I am wondering how we should treat the
case where we finish by not compressing a group of blocks as there is
a safety fallback to not enforce a failure if a block cannot be
compressed.  Should we move back to the compression of individual
blocks or just log all those pages uncompressed without their holes?
I really don't expect a group of blocks to not be compressed, just
being a bit paranoid here about the fallback we'd better have.
--
Michael

Commits

  1. Add more error context to RestoreBlockImage() and consume it

  2. Add support for LZ4 with compression of full-page writes in WAL

  3. Extended statistics on expressions

  4. Be clear about whether a recovery pause has taken effect.

  5. Add GUC to enable compression of full page images stored in WAL.