Horton

Re: Introduce pg_receivewal gzip compression tests

Michael Paquier <michael@paquier.xyz>

View thread View on postgr.es
From: Michael Paquier <michael@paquier.xyz>
To: Georgios <gkokolatos@protonmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2021-07-12T06:42:15Z
Lists: pgsql-hackers 
On Fri, Jul 09, 2021 at 11:26:58AM +0000, Georgios wrote:
> As suggested on a different thread [1], pg_receivewal can increase it's test
> coverage. There exists a non trivial amount of code that handles gzip
> compression. The current patch introduces tests that cover creation of gzip
> compressed WAL files and the handling of gzip partial segments. Finally the
> integrity of the compressed files is verified.

+       # Verify compressed file's integrity
+       my $gzip_is_valid = system_log('gzip', '--test', $gzip_wals[0]);
+       is($gzip_is_valid, 0, "program gzip verified file's integrity");
libz and gzip are usually split across different packages, hence there
is no guarantee that this command is always available (same comment as
for LZ4 from a couple of days ago).

+               [
+                       'pg_receivewal', '-D',     $stream_dir, '--verbose',
+                       '--endpos',      $nextlsn, '-Z', '5'
+               ],
I would keep the compression level to a minimum here, to limit CPU
usage but still compress something faster.

+       # Verify compressed file's integrity
+       my $gzip_is_valid = system_log('gzip', '--test', $gzip_wals[0]);
+       is($gzip_is_valid, 0, "program gzip verified file's integrity");
Shouldn't this be coded as a loop going through @gzip_wals?
--
Michael

Commits

  1. Re-enable TAP tests of pg_receivewal for ZLIB on Windows

  2. Fix some issues with WAL segment opening for pg_receivewal --compress

  3. Disable tests involving ZLIB on Windows for pg_receivewal

  4. Fix portability issue with gzip in TAP test of pg_receivewal

  5. Add TAP tests for ZLIB compression for pg_receivewal