Horton

Re: Cirrus-ci is lowering free CI cycles - what to do with cfbot, etc?

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Peter Eisentraut <peter@eisentraut.org>, pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@gmail.com>, Justin Pryzby <pryzby@telsasoft.com>
Date: 2023-08-08T20:44:52Z
Lists: pgsql-hackers 
Hi,

On 2023-08-08 18:34:58 +0200, Alvaro Herrera wrote:
> On 2023-Aug-08, Andres Freund wrote:
> 
> > Given the cost of macos, it seems like it'd be by far the most of affordable
> > to just buy 1-2 mac minis (2x ~660USD) and stick them in a shelf somewhere, as
> > persistent runners. Cirrus has builtin macos virtualization support - but can
> > only host two VMs on each mac, due to macos licensing restrictions. A single
> > mac mini would suffice to keep up with our unoptimized monthly runtime
> > (although there likely would be some overhead).
> 
> If using persistent workers is an option, maybe we should explore that.
> I think we could move all or some of the Linux - Debian builds to
> hardware that we already have in shelves (depending on how much compute
> power is really needed.)

(76+830+860+935)/((365/12)*24) = 3.7

3.7 instances with 4 "vcores" are busy 100% of the time. So we'd need at least
~16 cpu threads - I think cirrus sometimes uses instances that disable HT, so
it'd perhaps be 16 cores actually.


> I think using other OSes is more difficult, mostly because I doubt we
> want to deal with licenses; but even FreeBSD might not be a realistic
> option, at least not in the short term.

They can be VMs, so that shouldn't be a big issue.

> >                    task_name                    |    sum
> > ------------------------------------------------+------------
> >  FreeBSD - 13 - Meson                           | 1017:56:09
> >  Windows - Server 2019, MinGW64 - Meson         | 00:00:00
> >  SanityCheck                                    | 76:48:41
> >  macOS - Ventura - Meson                        | 873:12:43
> >  Windows - Server 2019, VS 2019 - Meson & ninja | 1251:08:06
> >  Linux - Debian Bullseye - Autoconf             | 830:17:26
> >  Linux - Debian Bullseye - Meson                | 860:37:21
> >  CompilerWarnings                               | 935:30:35
> > (8 rows)
> >
> 
> moving just Debian, that might alleviate 76+830+860+935 hours from the
> Cirrus infra, which is ~46%.  Not bad.
> 
> 
> (How come Windows - Meson reports allballs?)

It's mingw64, which we've marked as "manual", because we didn't have the cpu
cycles to run it.

Greetings,

Andres Freund

Commits

  1. Use snprintf instead of sprintf in pg_regress.

  2. Speed up pg_regress server readiness testing.

  3. ci: Make compute resources for CI configurable

  4. ci: Prepare to make compute resources for CI configurable

  5. ci: Use VMs for SanityCheck and CompilerWarnings

  6. ci: Move execution method of tasks into yaml templates

  7. ci: Don't specify amount of memory

  8. ci: macos: Remove use of -Dsegsize_blocks=6

  9. ci: macos: Remove use of -DRANDOMIZE_ALLOCATED_MEMORY