Re: Different compression methods for FPI

rebased to keep cfbot happy.  This will run with default=zlib.

On Mon, Mar 15, 2021 at 01:09:18PM -0500, Justin Pryzby wrote:
> On Sun, Mar 14, 2021 at 07:31:35PM -0500, Justin Pryzby wrote:
> > On Sat, Mar 13, 2021 at 08:48:33PM +0500, Andrey Borodin wrote:
> > > > 13 марта 2021 г., в 06:28, Justin Pryzby <pryzby@telsasoft.com> написал(а):
> > > > Updated patch with a minor fix to configure.ac to avoid warnings on OSX.
> > > > And 2ndary patches from another thread to allow passing recovery tests.
> > > > Renamed to WAL_COMPRESSION_*
> > > > Split LZ4 support to a separate patch and support zstd.  These show that
> > > > changes needed for a new compression method have been minimized, although not
> > > > yet moved to a separate, abstracted compression/decompression function.
> > > 
> > > Thanks! Awesome work!
> > > 
> > > > These two patches are a prerequisite for this patch to progress:
> > > > * Run 011_crash_recovery.pl with wal_level=minimal
> > > > * Make sure published XIDs are persistent
> > > > 
> > > > I don't know if anyone will consider this patch for v14 - if not, it should be
> > > > set to v15 and revisit in a month.  
> > > 
> > > I want to note, that fixes for 011_crash_recovery.pl are not strictly necessary for this patch set.
> > > The problem in tests arises if we turn on wal_compression, absolutely independently from wal compression method.
> > > We turn on wal_compression in this test only for CI purposes.
> > 
> > I rearranged the patches to reflect this.
> > Change to zlib and zstd to level=1.
> > Add support for negative "zstd fast" levels.
> > Use correct length accounting for "hole" in LZ4 and ZSTD.
> > Fixed Solution.pm for zstd on windows.
> > Switch to zstd by default (for CI).
> > Add docs.
> 
> Changes:
> - Allocate buffer sufficient to accommodate any supported compression method;
> - Use existing info flags argument rather than adding another byte for storing
>   the compression method; this seems to be what was anticipated by commit
>   57aa5b2bb and what Michael objected to.
> 
> I think the existing structures are ugly, so maybe this suggests using a GUC
> assign hook to support arbitrary compression level, and maybe other options.