Re: Introduce pg_receivewal gzip compression tests

On Fri, Jul 16, 2021 at 08:59:11AM +0900, Michael Paquier wrote:
> --compress is used and the sync fails for a non-compressed segment.
> Looking at the code it is pretty obvious that open_walfile() is
> getting confused with the handling of an existing .partial segment
> while walmethods.c uses dir_data->compression in all the places that
> matter.  So that's a legit bug, that happens only when mixing
> pg_receivewal runs for where successive runs use the compression or
> non-compression modes.

Ditto.  After reading the code more carefully, the code is actually
able to work even if it could be cleaner:
1) dir_existsfile() would check for the existence of a
non-compressed, partial segment all the time.
2) If this non-compressed file was padded, the code would use
open_for_write() that would open a compressed, partial segment.
3) The compressed, partial segment would be the one flushed.

This behavior is rather debatable, and it would be more instinctive to
me to just skip any business related to the pre-padding if compression
is enabled, at the cost of one extra callback in WalWriteMethod to
grab the compression level (dir_open_for_write() skips that for
compression) to allow receivelog.c to handle that.  But at the same
time few users are going to care about that as pg_receivewal has most
likely always the same set of options, so complicating this code is
not really appealing either.

> I am amazed that the other buildfarm members are not complaining, to
> be honest.  jacana runs this TAP test with MinGW and ZLIB, and does
> not complain.

I have spent more time on that with my own environment, and while
testing I have bumped on a different issue with zlib, which was
really weird.  In the same scenario as above, gzdopen() has been
failing for me at step 2), causing the test to loop forever.  We
document to use DLLs for ZLIB coming from zlib.net, but the ones
available there are really outdated as far as I can see (found some
called zlib.lib/dll myself, breaking Solution.pm).  For now I have
disabled those tests on Windows to bring back bowerbird to green, but
there is something else going on here.  We don't do much tests with
ZLIB on Windows for pg_basebackup and pg_dump, so there may be some
more issues?

@Andrew: which version of ZLIB are you using on bowerbird?  That's the
one in c:\prog\3p64.  That's a zdll.lib, right?
--
Michael