Horton
Menu

Re: Adding CI to our tree

Daniel Gustafsson <daniel@yesql.se>

View thread
From: Daniel Gustafsson <daniel@yesql.se>
To: Andres Freund <andres@anarazel.de>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>, Thomas Munro <thomas.munro@gmail.com>, Andrew Dunstan <andrew@dunslane.net>
Date: 2021-10-02T18:42:00Z
Lists: pgsql-hackers 
> On 2 Oct 2021, at 00:27, Andres Freund <andres@anarazel.de> wrote:

> For several development efforts I found it to be incredibly valuable to push
> changes to a personal repository and see a while later whether tests succeed
> on a number of different platforms.  This is especially useful for platforms
> that are quite different from ones own platform, like e.g. windows in my case.

Same, and for my case I run several CI jobs to compile/test against different
OpenSSL versions etc.

> Of course everybody can set this up for themselves. However, doing so well is
> a significant effort, particularly if windows is to be supported well. And
> doubly so if useful things like getting backtraces for crashes is desirable
> ([1])

+1 on adding these, rather than having everyone duplicate the effort.  Those
who don't want to use them can disregard them.

> Right now the patch attached
> - runs check-world on FreeBSD, Linux, macOS - all using gcc
>  - freebsd, linux use a custom generated image
>  - macOS installs missing dependencies at runtime, with some caching
>  - all use ccache to make subsequent compilation faster
> - runs all the tests I could find on windows, via vcregress.pl
> - checks for compiler warnings on linux, with both clang and gcc

Why not compiling with OpenSSL on FreeBSD and macOS?  On FreeBSD all you need
is --with-ssl=openssl while on macOS you need to point to the headers and libs
like:

  --with-includes=/usr/local/include:/usr/local/opt/openssl/include --with-libs=/usr/local/libs:/usr/local/opt/openssl/lib

One thing to note for Cirrus on macOS (I've never seen it anywhere else) is
that it intermittently will fail on a too long socketpath:

  Unix-domain socket path "/private/var/folders/wh/z5_y2cv53sg24tzvtw_f_y1m0000gn/T/cirrus-ci-build/src/bin/pg_upgrade/.s.PGSQL.51696" is too long (maximum 103 bytes)

Exporting PGSOCKETDIR can avoid that annoyance.

+  tests_script:
+    - su postgres -c 'ulimit -c unlimited ; ${TIMEOUT_CMD} make -s ${CHECK} ${CHECKFLAGS} -j8'
Don't you need PG_TEST_EXTRA=ssl here to ensure the src/test/ssl tests are run?

--
Daniel Gustafsson		https://vmware.com/

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. ci: enable zstd where available.

  2. ci: compile with -Og where applicable.

  3. ci: include hints how to install OS packages.

  4. ci: fix copy-paste mistake in 16eb8231d1b.

  5. ci: macos: align sysinfo_script to other tasks.

  6. ci: Only use one artifact instruction for logs.

  7. ci: s/CCACHE_SIZE/CCACHE_MAXSIZE/.

  8. pg_basebackup: Skip a few more fsyncs if --no-sync is specified.

  9. TAP tests: check for postmaster.pid anyway when "pg_ctl start" fails.

  10. Don't enable fsync in src/test/recovery/t/008_fsm_truncation.pl.

  11. ci: windows: run initdb with --no-sync.

  12. ci: windows: enable build summary to make it easier to spot warnings / errors.

  13. ci: Add continuous integration for github repositories via cirrus-ci.

  14. Fix TestLib::slurp_file() with offset on windows.