Horton
Menu

Re: CI and test improvements

Justin Pryzby <pryzby@telsasoft.com>

View thread
From: Justin Pryzby <pryzby@telsasoft.com>
To: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Cc: Thomas Munro <thomas.munro@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Andres Freund <andres@anarazel.de>, Andrew Dunstan <andrew@dunslane.net>, pgsql-hackers@lists.postgresql.org, Noah Misch <noah@leadboat.com>, Michael Paquier <michael@paquier.xyz>, Anastasia Lubennikova <lubennikovaav@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, Daniel Gustafsson <daniel@yesql.se>, samay sharma <smilingsamay@gmail.com>
Date: 2023-03-14T04:56:56Z
Lists: pgsql-hackers

Attachments

On Mon, Mar 13, 2023 at 07:39:52AM +0100, Peter Eisentraut wrote:
> On 03.02.23 15:26, Justin Pryzby wrote:
> > rebased, and re-including a patch to show code coverage of changed
> > files.
> 
> This constant flow of patches under one subject doesn't lend itself well to
> the commit fest model of trying to finish things up.
> I can't quite tell which of these patches are ready and agreed upon,
> and which ones are work in progress or experimental.

I'm soliticing feedback on those patches that I've sent recently - I've
elided patches if they have some unresolved issue.

I'm not aware of any loose ends other than what's updated here:

- cirrus: code coverage

I changed this to also run an "initial" coverage report before running
tests.  It's not clear to me what effect that has, though...

Andres seems to think it's a problem that this shows coverage only for
files that were actually changed.  But that's what's intended; it's
sufficient to see if new code is being hit by tests.  It would be slow
and take a lot of extra space to upload a coverage report for every
patch, every day.  It might be nice for cfbot to show how test coverage
changed in the affected files: -15% / +25%.

- cirrus: upload changed html docs as artifacts

Fixed an "only_if" line so cfbot will run the "warnings" task.

Maybe this path is waiting on Andres' patch to "move CompilerWarnings to
meson" ?

> > 7e09035f588 WIP: ci/meson: allow showing only failed tests ..
> 
> I'm not sure I like this one.  I sometimes look up the logs of non-failed
> tests to compare them with failed tests, to get context to could lead to
> failures.  Maybe we can make this behavior adjustable. But I've not been
> bothered by the current behavior.

It's adjustable by un/setting the environment variable.

I'm surprised to hear that anyone using cirrusci (with or without cfbot)
wouldn't prefer the behavior this patch implements.  It's annoying to
search find the logs for the (typically exactly one) failing test in
cirrus' directory of 200some test artifacts.  We're also uploading a lot
of logs for every failure.  (But I suppose this might break cfbot's new
client side parsing of things like build logs...)

-- 
Justin

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. ci: Add test coverage of different pg_upgrade modes

  2. seg: Add test "security" in meson.build

  3. cirrus/freebsd: define ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS

  4. cirrus/ccache: Use G rather than GB suffix

  5. ci: Upgrade macOS version from 12 to 13.

  6. meson: Add two missing regress tests

  7. Push lpp variable closer to usage in heapgetpage()

  8. ci: Change macOS builds from Intel to ARM.

  9. ci: Introduce SanityCheck task that other tasks depend on

  10. ci: Use -fsanitize=undefined,alignment,address in linux tasks

  11. ci: Clean up pre-meson cruft in windows task

  12. meson: Mark PROVE as not required

  13. ci: enable various runtime checks on FreeBSD and macOS