Re: meson: Make test output much more useful on failure (both in CI and locally)
Corey Huinker <corey.huinker@gmail.com>
From: Corey Huinker <corey.huinker@gmail.com>
To: Jelte Fennema-Nio <postgres@jeltef.nl>
Cc: Zsolt Parragi <zsolt.parragi@percona.com>,
Nazir Bilal Yavuz <byavuz81@gmail.com>, Andres Freund <andres@anarazel.de>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, andrew@dunslane.net
Date: 2026-03-19T19:44:17Z
Lists: pgsql-hackers
> > > Finally I added a new commit which starts to use command_ok in > 002_pg_upgrade.pl and 027_stream_regress.pl to get the new nicer output. > I tried this out, and am very pleased with the results. I added an obvious error to a regression output file and, now I see: # -------------- stderr -------------- # # diff -U3 /home/corey/src/postgres/src/test/regress/expected/stats_import.out /home/corey/src/postgres/build/testrun/recovery/027_stream_regress/data/results/stats_import.out # # --- /home/corey/src/postgres/src/test/regress/expected/stats_import.out 2026-03-19 15:20:35.413217825 -0400 # # +++ /home/corey/src/postgres/build/testrun/recovery/027_stream_regress/data/results/stats_import.out 2026-03-19 15:20:57.565988863 -0400 # # @@ -25,8 +25,6 @@ # # 31 # # (1 row) # # # # --- OBVIOUS ERROR IS OBVIOUS # # - # # -- Create a view that is used purely for the type based on pg_statistic. # # CREATE VIEW stats_import.pg_statistic_flat_t AS # # SELECT # # 1 of 243 tests failed. # # The differences that caused some tests to fail can be viewed in the file "/home/corey/src/postgres/build/testrun/recovery/027_stream_regress/data/regression.diffs". # # A copy of the test summary that you see above is saved in the file "/home/corey/src/postgres/build/testrun/recovery/027_stream_regress/data/regression.out". # ------------------------------------ # Looks like you failed 1 test of 11. (test program exited with status code 1) ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― That's great, because ofttimes I want to find the diff -U3 line, replace the command with "meld" (a graphical diff tool) and then examine the differences from there. v4-0001: + * useful information is actually in the first few lines nitpick: comment sentence needs a closing '.' v4-0002, v4-0003, v4-0004: look good to me v4-0005: the commit message is probably too bulky to be used as-is, but the committer was going to change that anyway and right now too much information is way better than not enough. I can't wait to use this.
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Use command_ok for pg_regress calls in 002_pg_upgrade and 027_stream_regress
- bb6ae9707c17 19 (unreleased) landed
-
perl tap: Show die reason in TAP output
- 76540fdedf40 19 (unreleased) landed
-
perl tap: Show failed command output
- 1402b8d2fc70 19 (unreleased) landed
-
perl tap: Use croak instead of die in our helper modules
- b8da9869b850 19 (unreleased) landed
-
pg_regress: Include diffs in TAP output
- 5720ae014364 19 (unreleased) landed