Re: meson: Make test output much more useful on failure (both in CI and locally)

Corey Huinker <corey.huinker@gmail.com>

From: Corey Huinker <corey.huinker@gmail.com>
To: Jelte Fennema-Nio <postgres@jeltef.nl>
Cc: Zsolt Parragi <zsolt.parragi@percona.com>, Nazir Bilal Yavuz <byavuz81@gmail.com>, Andres Freund <andres@anarazel.de>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, andrew@dunslane.net
Date: 2026-03-19T19:44:17Z
Lists: pgsql-hackers
>
>
> Finally I added a new commit which starts to use command_ok in
> 002_pg_upgrade.pl and 027_stream_regress.pl to get the new nicer output.
>

I tried this out, and am very pleased with the results.

I added an obvious error to a regression output file and, now I see:

# -------------- stderr --------------
# # diff -U3
/home/corey/src/postgres/src/test/regress/expected/stats_import.out
/home/corey/src/postgres/build/testrun/recovery/027_stream_regress/data/results/stats_import.out
# # --- /home/corey/src/postgres/src/test/regress/expected/stats_import.out
2026-03-19 15:20:35.413217825 -0400
# # +++
/home/corey/src/postgres/build/testrun/recovery/027_stream_regress/data/results/stats_import.out
2026-03-19 15:20:57.565988863 -0400
# # @@ -25,8 +25,6 @@
# #      31
# #  (1 row)
# #
# # --- OBVIOUS ERROR IS OBVIOUS
# # -
# #  -- Create a view that is used purely for the type based on
pg_statistic.
# #  CREATE VIEW stats_import.pg_statistic_flat_t AS
# #    SELECT
# # 1 of 243 tests failed.
# # The differences that caused some tests to fail can be viewed in the
file
"/home/corey/src/postgres/build/testrun/recovery/027_stream_regress/data/regression.diffs".
# # A copy of the test summary that you see above is saved in the file
"/home/corey/src/postgres/build/testrun/recovery/027_stream_regress/data/regression.out".
# ------------------------------------
# Looks like you failed 1 test of 11.

(test program exited with status code 1)
―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――


That's great, because ofttimes I want to find the diff -U3 line, replace
the command with "meld" (a graphical diff tool) and then examine the
differences from there.

v4-0001:

+ * useful information is actually in the first few lines

nitpick: comment sentence needs a closing '.'

v4-0002, v4-0003, v4-0004: look good to me

v4-0005: the commit message is probably too bulky to be used as-is, but the
committer was going to change that anyway and right now too much
information is way better than not enough.

I can't wait to use this.

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Use command_ok for pg_regress calls in 002_pg_upgrade and 027_stream_regress

  2. perl tap: Show die reason in TAP output

  3. perl tap: Show failed command output

  4. perl tap: Use croak instead of die in our helper modules

  5. pg_regress: Include diffs in TAP output