Re: Teach pg_receivewal to use lz4 compression

gkokolatos@pm.me

From: gkokolatos@pm.me
To: Michael Paquier <michael@paquier.xyz>
Cc: Dilip Kumar <dilipbalaut@gmail.com>, Magnus Hagander <magnus@hagander.net>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2021-07-12T09:33:44Z
Lists: pgsql-hackers

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Monday, July 12th, 2021 at 07:56, Michael Paquier <michael@paquier.xyz> wrote:

> On Mon, Jul 12, 2021 at 11:10:24AM +0530, Dilip Kumar wrote:
>
> > On Thu, Jul 8, 2021 at 7:48 PM gkokolatos@pm.me wrote:
> >
> > > We can, though I am not in favour of doing so. There is seemingly
> > >
> > > little benefit for added complexity.
> >
> > I am really not sure what complexity you are talking about, do you
> >
> > mean since with pglz we were already providing the compression level
> >
> > so let it be as it is but with the new compression method you don't
> >
> > see much benefit of providing compression level or speed?
>
> You mean s/pglz/zlib/ here perhaps? I am not sure what Georgios has
>
> in mind, but my opinion stands on the latter: there is little benefit
>
> in making lz4 faster than the default and reduce compression, as the
>
> default is already a rather low CPU user.

Thank you all for your comments. I am sitting on the same side as Micheal
on this one. The complexity is not huge, yet there will need to be code to
pass this option to the lz4 api and various test cases to verify for
correctness and integrity. The burden of maintenance of this code vs the
benefit of the option, tilt the scale towards not including the option.

Of course, I will happily provide whatever the community finds beneficial.

Cheers,
//Georgios


> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Michael



Commits

  1. Fix header inclusion order in pg_receivewal.c

  2. Remove useless LZ4 system call on failure when writing file header

  3. Add support for LZ4 compression in pg_receivewal

  4. Rework compression options of pg_receivewal

  5. Clarify some errors in pg_receivewal when closing WAL segments