Horton
Menu

Re: CI and test improvements

Thomas Munro <thomas.munro@gmail.com>

View thread
From: Thomas Munro <thomas.munro@gmail.com>
To: Justin Pryzby <pryzby@telsasoft.com>
Cc: Andres Freund <andres@anarazel.de>, Andrew Dunstan <andrew@dunslane.net>, pgsql-hackers@lists.postgresql.org, Noah Misch <noah@leadboat.com>, Michael Paquier <michael@paquier.xyz>, Anastasia Lubennikova <lubennikovaav@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Robert Haas <robertmhaas@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, Daniel Gustafsson <daniel@yesql.se>
Date: 2022-09-23T03:32:48Z
Lists: pgsql-hackers 
On Fri, Sep 23, 2022 at 9:07 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
> On Sun, Aug 28, 2022 at 02:28:02PM -0700, Andres Freund wrote:
> > > > > -  # Workaround around performance issues due to 32KB block size
> > > > > -  repartition_script: src/tools/ci/gcp_freebsd_repartition.sh
> > > > >    create_user_script: |
> > > > >      pw useradd postgres
> > > > >      chown -R postgres:postgres .
> > > >
> > > > What's the story there - at some point that was important for performance
> > > > because of the native block size triggering significant read-modify-write
> > > > cycles with postres' writes. You didn't comment on it in the commit message.
> > >
> > > Well, I don't know the history, but it seems to be unneeded now.
> >
> > It's possible it was mainly needed for testing with aio + dio. But also
> > possible that an upgrade improved the situation since.

Yeah, it is very important for direct I/O (patches soon...), because
every 8KB random write becomes a read-32KB/wait/write-32KB without it.
It's slightly less important for buffered I/O, because the kernel
caches hide that, but it still triggers I/O bandwidth amplification,
and we definitely saw positive effects earlier on the CI system back
on the previous generation.  FWIW I am planning to see about getting
the FreeBSD installer to create the root file system the way we want,
instead of this ugliness.

> Maybe freebsd got faster as a result of the TAU CPUs?
> [data]

Very interesting.  Would be good to find the exact instance types +
storage types to see if there has been a massive IOPS boost, perhaps
via local SSD.  The newer times are getting closer to a local
developer machine.

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. ci: Add test coverage of different pg_upgrade modes

  2. seg: Add test "security" in meson.build

  3. cirrus/freebsd: define ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS

  4. cirrus/ccache: Use G rather than GB suffix

  5. ci: Upgrade macOS version from 12 to 13.

  6. meson: Add two missing regress tests

  7. Push lpp variable closer to usage in heapgetpage()

  8. ci: Change macOS builds from Intel to ARM.

  9. ci: Introduce SanityCheck task that other tasks depend on

  10. ci: Use -fsanitize=undefined,alignment,address in linux tasks

  11. ci: Clean up pre-meson cruft in windows task

  12. meson: Mark PROVE as not required

  13. ci: enable various runtime checks on FreeBSD and macOS