Horton

Re: pg_basebackup's --gzip switch misbehaves

Justin Pryzby <pryzby@telsasoft.com>

View thread View on postgr.es
From: Justin Pryzby <pryzby@telsasoft.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers@lists.postgresql.org
Date: 2022-09-22T00:31:48Z
Lists: pgsql-hackers 
On Tue, Sep 13, 2022 at 04:13:20PM +0900, Michael Paquier wrote:
> diff --git a/src/common/compression.c b/src/common/compression.c
> index da3c291c0f..ac26287d54 100644
> --- a/src/common/compression.c
> +++ b/src/common/compression.c
> @@ -249,36 +299,49 @@ expect_integer_value(char *keyword, char *value, pg_compress_specification *resu
>  char *
>  validate_compress_specification(pg_compress_specification *spec)
>  {
> +	int			min_level = 1;
> +	int			max_level = 1;
> +	int			default_level = 0;
> +
>  	/* If it didn't even parse OK, it's definitely no good. */
>  	if (spec->parse_error != NULL)
>  		return spec->parse_error;
>  
>  	/*
> -	 * If a compression level was specified, check that the algorithm expects
> -	 * a compression level and that the level is within the legal range for
> -	 * the algorithm.
> +	 * Check that the algorithm expects a compression level and it is
> +	 * is within the legal range for the algorithm.
>  	 */
> -	if ((spec->options & PG_COMPRESSION_OPTION_LEVEL) != 0)
> +	switch (spec->algorithm)
>  	{
> -		int			min_level = 1;
> -		int			max_level;
> -
> -		if (spec->algorithm == PG_COMPRESSION_GZIP)
> +		case PG_COMPRESSION_GZIP:
>  			max_level = 9;
> -		else if (spec->algorithm == PG_COMPRESSION_LZ4)
> +#ifdef HAVE_LIBZ
> +			default_level = Z_DEFAULT_COMPRESSION;
> +#endif
> +			break;
> +		case PG_COMPRESSION_LZ4:
>  			max_level = 12;
> -		else if (spec->algorithm == PG_COMPRESSION_ZSTD)
> +			default_level = 0;	/* fast mode */
> +			break;
> +		case PG_COMPRESSION_ZSTD:
>  			max_level = 22;

I should've suggested to add:

>  			min_level = -7;

which has been supported since zstd 1.3.4 (and postgres requires 1.4.0).

I think at some point (maybe before releasing 1.3.4) the range was
increased to very large(small), negative levels.  It's possible to query
the library about the lowest supported compression level, but then
there's a complication regarding the client-side library version vs the
server-side version.  So it seems better to just use -7.

-- 
Justin

Commits

  1. Make PG_TEST_NOCLEAN work for temporary directories in TAP tests

  2. doc: document the TAP test environment variables

  3. doc: update metacpan.org links to avoid redirects

  4. Use min/max bounds defined by Zstd for compression level

  5. Fix incorrect value for "strategy" with deflateParams() in walmethods.c

  6. Simplify handling of compression level with compression specifications