Horton
Menu

Adding CI to our tree

Andres Freund <andres@anarazel.de>

View thread
From: Andres Freund <andres@anarazel.de>
To: pgsql-hackers@postgresql.org
Cc: Thomas Munro <thomas.munro@gmail.com>, Andrew Dunstan <andrew@dunslane.net>
Date: 2021-10-01T22:27:52Z
Lists: pgsql-hackers

Attachments

Hi,

For several development efforts I found it to be incredibly valuable to push
changes to a personal repository and see a while later whether tests succeed
on a number of different platforms.  This is especially useful for platforms
that are quite different from ones own platform, like e.g. windows in my case.

Of course everybody can set this up for themselves. However, doing so well is
a significant effort, particularly if windows is to be supported well. And
doubly so if useful things like getting backtraces for crashes is desirable
([1])

We do a form of pre-commit CI via cfbot. That is valuable. But it's not really
comparable to having CI in tree - one need to post to the list and one cannot
adjust the dependencies etc installed for the CI runs.


New contributors (and quite a bit of older ones too) IMO expect to be able to
see whether their changes work as-is, without sending a patch to the list.


An obvious criticism of the effort to put CI runner infrastructure into core
is that they are effectively all proprietary technology, and that we should be
hesistant to depend too much on one of them. I think that's a valid
concern. However, once one CI integration is done, a good chunk (but not all!)
the work is transferrable to another CI solution, which I do think reduces the
dependency sufficiently.


The attached patch adds CI using cirrus-ci. The reason for choosing cirrus
were that
a) Thomas has ended up using cirrus for cfbot
b) cirrus provides a comparatively wide variety of operating systems
c) it allows custom VM images to be used.
d) it does not require a login to look at

c) is very valuable to be able to test e.g. upcoming linux versions,
pre-installing software on systems that do not support docker (freebsd), and
being faster to boot once the image is more than a trivial size.  I've created
a number of images for testing of the aio patchset [2]


Right now the patch attached
- runs check-world on FreeBSD, Linux, macOS - all using gcc
  - freebsd, linux use a custom generated image
  - macOS installs missing dependencies at runtime, with some caching
  - all use ccache to make subsequent compilation faster
- runs all the tests I could find on windows, via vcregress.pl
- checks for compiler warnings on linux, with both clang and gcc

- captures all logs after a failing run
- generates backtraces from core files (the output format differs between platforms)
- allows to limit CI to certain OSs, by adding
  ci-os-only: (freebsd|linux|macos|windows)+ to the commit message
  (useful when fixing a platform dependent problem)

Example output of a
- successful run: https://cirrus-ci.com/build/4625606928236544
- github interface for the same: https://github.com/anarazel/postgres/runs/3772435617
- failed run on windows, with backtrace: https://cirrus-ci.com/task/6640561307254784?logs=cat_dumps#L150

Comments? Rotten tomatoes?


There's some polishing we should do before actually adding this to the
tree. But I wanted to discuss the idea before investing even more time.

One policy discussion that we'd have to have is who should control the images
used for CI. Right now that's on my personal google cloud account - which I am
happy to do, but medium - long term that'd not be optimal.


Thanks to Thomas - I based this on hist .cirrus.yml file. And made several
contributions later. Also thanks to Andrew, who helped out with some windows
issues I hit at some point...

Greetings,

Andres Freund

[1] I did get this to work on windows, but it does require a small set of
changes to work reliably, I'll start a separate thread about it.
[2] https://github.com/anarazel/pg-vm-images/

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. ci: enable zstd where available.

  2. ci: compile with -Og where applicable.

  3. ci: include hints how to install OS packages.

  4. ci: fix copy-paste mistake in 16eb8231d1b.

  5. ci: macos: align sysinfo_script to other tasks.

  6. ci: Only use one artifact instruction for logs.

  7. ci: s/CCACHE_SIZE/CCACHE_MAXSIZE/.

  8. pg_basebackup: Skip a few more fsyncs if --no-sync is specified.

  9. TAP tests: check for postmaster.pid anyway when "pg_ctl start" fails.

  10. Don't enable fsync in src/test/recovery/t/008_fsm_truncation.pl.

  11. ci: windows: run initdb with --no-sync.

  12. ci: windows: enable build summary to make it easier to spot warnings / errors.

  13. ci: Add continuous integration for github repositories via cirrus-ci.

  14. Fix TestLib::slurp_file() with offset on windows.