Horton

Thread

Commits

  1. Disable temporarily TAP tests for contrib/bloom/

  2. Add PGXS options to control TAP and isolation tests

  1. pgsql: Add PGXS options to control TAP and isolation tests

    Michael Paquier <michael@paquier.xyz> — 2018-11-25T23:53:02Z

    Add PGXS options to control TAP and isolation tests

The following options are added for extensions:
- TAP_TESTS, to allow an extention to run TAP tests which are the ones
present in t/*.pl.  A subset of tests can always be run with the
existing PROVE_TESTS for developers.
- ISOLATION, to define a list of isolation tests.
- ISOLATION_OPTS, to pass custom options to isolation_tester.

A couple of custom Makefile targets have been accumulated across the
tree to cover the lack of facility in PGXS for a couple of releases when
using those test suites, which are all now replaced with the new flags,
without reducing the test coverage.  This also fixes an issue with
contrib/bloom/, which had a custom target to trigger its TAP tests of
its own not part of the main check runs.

Author: Michael Paquier
Reviewed-by: Adam Berlin, Álvaro Herrera, Tom Lane, Nikolay Shaplov,
Arthur Zakirov
Discussion: https://postgr.es/m/20180906014849.GG2726@paquier.xyz

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/03faa4a8ddf04334850bf9631f305bda7a02f260

Modified Files
--------------
contrib/bloom/Makefile                     |  4 +-
contrib/oid2name/Makefile                  | 10 +----
contrib/test_decoding/Makefile             | 67 ++++++++----------------------
contrib/vacuumlo/Makefile                  | 10 +----
doc/src/sgml/extend.sgml                   | 61 ++++++++++++++++++++++++++-
src/makefiles/pgxs.mk                      | 32 +++++++++++++-
src/test/modules/brin/.gitignore           |  2 +-
src/test/modules/brin/Makefile             | 25 ++---------
src/test/modules/commit_ts/Makefile        |  6 +--
src/test/modules/snapshot_too_old/Makefile | 35 ++++------------
src/test/modules/test_pg_dump/Makefile     |  6 +--
11 files changed, 125 insertions(+), 133 deletions(-)

  2. Re: pgsql: Add PGXS options to control TAP and isolation tests

    Michael Paquier <michael@paquier.xyz> — 2018-11-26T00:33:51Z

    On Sun, Nov 25, 2018 at 11:53:02PM +0000, Michael Paquier wrote:
> Add PGXS options to control TAP and isolation tests
> 
> The following options are added for extensions:
> - TAP_TESTS, to allow an extention to run TAP tests which are the ones
> present in t/*.pl.  A subset of tests can always be run with the
> existing PROVE_TESTS for developers.
> - ISOLATION, to define a list of isolation tests.
> - ISOLATION_OPTS, to pass custom options to isolation_tester.
> 
> A couple of custom Makefile targets have been accumulated across the
> tree to cover the lack of facility in PGXS for a couple of releases when
> using those test suites, which are all now replaced with the new flags,
> without reducing the test coverage.  This also fixes an issue with
> contrib/bloom/, which had a custom target to trigger its TAP tests of
> its own not part of the main check runs.

Several buildfarm members complain about this commit, including dory and
longfin:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dory&dt=2018-11-26%2000%3A00%3A26
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=longfin&dt=2018-11-25%2023%3A59%3A03

The error comes from the TAP tests of contrib/bloom/, which we have
never run across the buildfarm until now:
#   Failed test 'initial: query result matches'
#   at t/001_wal.pl line 38.
#          got: '0|d
# 0|9
# 0|3

I am not sure what to think of that yet, so as a short-term fix and to
keep the buildfarm green, I am going to disable the TAP tests for bloom,
but something is busted there, either in the code or in the tests.
--
Michael

  3. Re: pgsql: Add PGXS options to control TAP and isolation tests

    Michael Paquier <michael@paquier.xyz> — 2018-11-26T02:21:15Z

    On Mon, Nov 26, 2018 at 09:33:51AM +0900, Michael Paquier wrote:
> Several buildfarm members complain about this commit, including dory and
> longfin:
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dory&dt=2018-11-26%2000%3A00%3A26
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=longfin&dt=2018-11-25%2023%3A59%3A03
> 
> The error comes from the TAP tests of contrib/bloom/, which we have
> never run across the buildfarm until now:
> #   Failed test 'initial: query result matches'
> #   at t/001_wal.pl line 38.
> #          got: '0|d
> # 0|9
> # 0|3
> 
> I am not sure what to think of that yet, so as a short-term fix and to
> keep the buildfarm green, I am going to disable the TAP tests for bloom,
> but something is busted there, either in the code or in the tests.

I have been fighting with the buildfarm a bit this morning, but this is
proving to have a larger subset of issues than I anticipated first.
Here is the collection of my notes:
- MSVC scripts assume that REGRESS_OPTS can only use top_builddir.  Some
test suites actually finish by using top_srcdir, like pg_stat_statements
which cause the regression tests to never run on Windows!
- Trying to enforce top_builddir does not work either when using VPATH
as this is not recognized properly.
- TAP tests of bloom are unstable on various platforms, causing various
failures.

As a result, I have reverted all the recent changes which introduced
those new PGXS options.  I am going to spawn a set of thread to address
all the underlying issues first as those deserve more discussion.  There
is a collection of tests not actually running on Windows.
--
Michael