Horton

Re: pg_upgrade test for binary compatibility of core data types

Jacob Champion <pchampion@vmware.com>

View thread
From: Jacob Champion <pchampion@vmware.com>
To: "pryzby@telsasoft.com" <pryzby@telsasoft.com>, "tgl@sss.pgh.pa.us" <tgl@sss.pgh.pa.us>
Cc: "peter.eisentraut@enterprisedb.com" <peter.eisentraut@enterprisedb.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, "buschmann@nidsa.net" <buschmann@nidsa.net>, "andrew@dunslane.net" <andrew@dunslane.net>, "noah@leadboat.com" <noah@leadboat.com>, "tomas.vondra@2ndquadrant.com" <tomas.vondra@2ndquadrant.com>, "bruce@momjian.us" <bruce@momjian.us>, "andres@anarazel.de" <andres@anarazel.de>
Date: 2021-07-16T18:02:18Z
Lists: pgsql-bugs, pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Move into separate file all the SQL queries used in pg_upgrade tests

  2. Add table to regression tests for binary-compatibility checks in pg_upgrade

  3. Fix tests of pg_upgrade across different major versions

  4. Multirange datatypes

  5. Work around cross-version-upgrade issues created by commit 9e38c2bb5.

  6. Declare assorted array functions using anycompatible not anyelement.

  7. Remove factorial operators, leaving only the factorial() function.

  8. Create by default sql/ and expected/ for output directory in pg_regress

  9. Add missing include to pg_upgrade/version.c

  10. Improve the check for pg_catalog.line data type in pg_upgrade

  11. Improve the check for pg_catalog.unknown data type in pg_upgrade

  12. Check for tables with sql_identifier during pg_upgrade

  13. pg_upgrade: clarify the database names in error files

  14. In the pg_upgrade test suite, don't write to src/test/regress.

  15. Allow group access on PGDATA

  16. Refactor dir/file permissions

  17. Remove unused functions in regress.c.

  18. Make WAL segment size configurable at initdb time.

  19. Fix bit-rot in pg_upgrade's test.sh, and improve documentation.

On Fri, 2021-04-30 at 13:33 -0500, Justin Pryzby wrote:
> On Sat, Mar 06, 2021 at 03:01:43PM -0500, Tom Lane wrote:
> > v4-0001 mostly teaches test.sh about specific changes that have to be
> > made to historic versions of the regression database to allow them
> > to be reloaded into current servers.  As already discussed, this is
> > really duplicative of knowledge that's been embedded into the buildfarm
> > client over time.  It'd be better if we could refactor that so that
> > the buildfarm shares a common database of these actions with test.sh.
> > And said database ought to be in our git tree, so committers could
> > fix problems without having to get Andrew involved every time.
> > I think this could be represented as a psql script, at least in
> > versions that have psql \if (but that came in in v10, so maybe
> > we're there already).
> 
> I started this.  I don't know if it's compatible with the buildfarm client, but
> I think any issues maybe can be avoided by using "IF EXISTS".

Here are the differences I see on a first pass (without putting too
much thought into how significant the differences are). Buildfarm code
I'm comparing against is at [1].

- Both versions drop @#@ and array_cat_accum, but the buildfarm
additionally replaces them with a new operator and aggregate,
respectively.

- The buildfarm's dropping of table OIDs is probably more resilient,
since it loops over pg_class looking for relhasoids.

- The buildfarm handles (or drops) several contrib databases in
addition to the core regression DB.

- The psql script drops the first_el_agg_any aggregate and a `TRANSFORM
FOR integer`; I don't see any corresponding code in the buildfarm.

- Some version ranges are different between the two. For example,
abstime_/reltime_/tinterval_tbl are dropped by the buildfarm if the old
version is < 9.3, while the psql script drops them for old versions <=
10.

- The buildfarm drops the public.=> operator for much older versions of
Postgres. I assume we don't need that here.

- The buildfarm adjusts pg_proc for the location of regress.so; I see
there's a commented placeholder for this at the end of the psql script
but it's not yet implemented.

As an aside, I think the "fromv10" naming scheme for the "old version
<= 10" condition is unintuitive. If the old version is e.g. 9.6, we're
not upgrading "from 10".

--Jacob

[1] https://github.com/PGBuildFarm/client-code/blob/main/PGBuild/Modules/TestUpgradeXversion.pm