Horton

Re: Test to dump and restore objects left behind by regression

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Daniel Gustafsson <daniel@yesql.se>, Tom Lane <tgl@sss.pgh.pa.us>, Michael Paquier <michael@paquier.xyz>, vignesh C <vignesh21@gmail.com>, Peter Eisentraut <peter@eisentraut.org>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2025-04-04T16:07:08Z
Lists: pgsql-hackers 
Hi,

On 2025-04-04 12:01:16 -0400, Andres Freund wrote:
> FWIW, for me 027 is actually considerably faster. In an cassert -O0 build (my
> normal development env, I find even -Og too problematic for debugging):
> 
> pg_upgrade/002_pg_upgrade    96.61s
> recovery/027_stream_regress  66.04s
> 
> After
>   git revert 8806e4e8deb1e21715e031e17181d904825a410e abe56227b2e213755dd3e194c530f5f06467bd7c 172259afb563d35001410dc6daad78b250924038
> 
> pg_upgrade/002_pg_upgrade    75.09s
> 
> Slower by 29%, far from the 3s increased time I saw mentioned somewhere.
> 
> 
> And this really affects the overall test time:
> 
> All tests before:
> 	real	1m38.173s
> 	user	5m52.500s
> 	sys	4m23.574s
> 
> All tests after:
> 	real	2m0.397s
> 	user	5m53.625s
> 	sys	4m30.518s
> 
> The CPU time increase is rather minimal, but the wall clock time increase is
> 22%.
> 
> 17:
> 	real	1m14.822s
> 	user	4m2.630s
> 	sys	3m22.384s
> 
> We regressed wall clock time *60%* from 17->18. Some test cycle increase is
> reasonable and can largely be compensated with hardware, but this cycle we're
> growing way faster than hardware gets faster.  I don't think that's
> sustainable.

FWIW, with cassert and -O2, it's:

17:
	real	0m53.981s
	user	3m22.837s
	sys	3m24.237s

HEAD:
	real	1m19.749s
	user	4m54.526s
	sys	4m15.657s

so this isn't just due to me using -O0. A 48% increase is better than a 60%
increase, but it's still not sustainable.

Greetings,

Andres Freund

Commits

  1. Hide expensive pg_upgrade test behind PG_TEST_EXTRA

  2. Set log_statement=none in t/002_pg_upgrade.pl

  3. 002_pg_upgrade.pl: Move pg_dump test code for better stability

  4. 002_pg_upgrade.pl: rename some variables for clarity

  5. Verify roundtrip dump/restore of regression database

  6. Refactor TAP test code for file comparisons into new routine in Utils.pm

  7. Virtual generated columns

  8. Put generated_stored test objects in a schema

  9. Introduce "builtin" collation provider.

  10. Revert "Improve compression and storage support with inheritance"