Horton
Menu

Re: Adding CI to our tree

Andres Freund <andres@anarazel.de>

View thread
From: Andres Freund <andres@anarazel.de>
To: 0010203112132233 <boekewurm@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Thomas Munro <thomas.munro@gmail.com>, Andrew Dunstan <andrew@dunslane.net>
Date: 2021-10-02T00:10:53Z
Lists: pgsql-hackers 
Hi,

On 2021-10-02 01:49:39 +0200, 0010203112132233 wrote:
> On Sat, 2 Oct 2021 at 00:28, Andres Freund <andres@anarazel.de> wrote:
> > New contributors (and quite a bit of older ones too) IMO expect to be able to
> > see whether their changes work as-is, without sending a patch to the list.
> 
> Have they checked 'make installcheck-world'? I believe that is one of
> the first action items on the 'So, you want to become a developer?'
> wiki page, after downloading the sources. Of course, that is limited
> to only the environment of the user, but that's what they're generally
> developing for, right?

If you want to get a change into postgres, it almost always needs to actually
work on all operating systems, and always needs to at least not cause build
failures on all platforms.



> Furthermore, after looking it through, I think that Cirrus is an
> unfortunate choice as a CI platform of preference, as you cannot use
> it without access to Github (which is problematic for people located
> in certain localities due to USA regulations).

I agree that it's not optimal that cirrus isn't available on all git hosting
platforms. Hence saying that I think it's likely we'd end up adding a few more
platforms over time.  If we factor the meat of the work into an helper script,
so that the CI specific bit is just a couple invocation of that script, it's
not a lot of overhead to have 2-3 CI platforms.


> If we're going to include CI configuration for private use, I'd prefer if it
> were a CI that can be enjoyed in private without pushing code to a 3rd
> party.

FWIW, you can use cirrus locally on your machine:
https://github.com/cirruslabs/cirrus-cli

It'll not be able to run all kinds of tasks though (e.g. no windows docker on
a linux host, dealing with the license costs for that presumably would be
nontrivial).


> Lastly, I consider CI configuration similar to IDE configuration: each
> developer has their own preferred tools which they use, but we don't
> favour one over the other. We don't include IDE-specific configuration
> files either, or at least, the policy is against that.
> 
> So, I greatly appreciate the effort, but I don't think this is
> something that should be committed into core. Maybe as a dedicated
> wiki page detailing configurations for CI, similar to the Buildfarm
> page?

That doesn't scale - I've actually added CI to all my substantial development
work in my private branches, and it's pretty annoying to need to do so every
time. And there's many developers who won't go through the effort most of the
time.

It's not like this forces you to use cirrus or anything. For people that don't
want to use CI, It'll make cfbot a bit more effective (because people can
adjust what it tests as appropriate for $patch), but that's it.

Greetings,

Andres Freund

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. ci: enable zstd where available.

  2. ci: compile with -Og where applicable.

  3. ci: include hints how to install OS packages.

  4. ci: fix copy-paste mistake in 16eb8231d1b.

  5. ci: macos: align sysinfo_script to other tasks.

  6. ci: Only use one artifact instruction for logs.

  7. ci: s/CCACHE_SIZE/CCACHE_MAXSIZE/.

  8. pg_basebackup: Skip a few more fsyncs if --no-sync is specified.

  9. TAP tests: check for postmaster.pid anyway when "pg_ctl start" fails.

  10. Don't enable fsync in src/test/recovery/t/008_fsm_truncation.pl.

  11. ci: windows: run initdb with --no-sync.

  12. ci: windows: enable build summary to make it easier to spot warnings / errors.

  13. ci: Add continuous integration for github repositories via cirrus-ci.

  14. Fix TestLib::slurp_file() with offset on windows.