Horton

Re: AdvanceXLInsertBuffer vs. WAL segment compressibility

Michael Paquier <michael.paquier@gmail.com>

View thread View on postgr.es
From: Michael Paquier <michael.paquier@gmail.com>
To: Chapman Flack <chap@anastigmatix.net>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>, Robert Haas <robertmhaas@gmail.com>, Bruce Momjian <bruce@momjian.us>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2017-07-17T15:00:02Z
Lists: pgsql-hackers 
On Fri, Jun 23, 2017 at 6:08 AM, Chapman Flack <chap@anastigmatix.net> wrote:
> Well, gzip was doing pretty well; it could get a 16 MB segment file down
> to under 27 kB, or less than 14 bytes for each of 2000 pages, when a page
> header is what, 20 bytes, it looks like? I'm not sure how much better
> I'd expect a (non-custom) compression scheme to do. The real difference
> comes between compressing (even well) a large unchanged area, versus being
> able to recognize (again with a non-custom tool) that the whole area is
> unchanged.

Have you tried as well lz4 for your cases? It performs faster than
gzip at minimum compression and compresses less, but I am really
wondering if for almost zero pages it performs actually better.
-- 
Michael

Commits

  1. Ensure that WAL pages skipped by a forced WAL switch are zero-filled.

  2. Improve scalability of WAL insertions.

  3. Move BKP_REMOVABLE bit from individual WAL records to WAL page headers.