Horton

Re: [RFC] building postgres with meson - v12

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Cc: pgsql-hackers@postgresql.org, samay sharma <smilingsamay@gmail.com>, Nazir Bilal Yavuz <byavuz81@gmail.com>
Date: 2022-08-31T18:11:54Z
Lists: pgsql-hackers 
Hi,

On 2022-08-31 10:28:05 +0200, Peter Eisentraut wrote:
> I found that the perl test modules are not installed.  See attached patch to
> correct this.
>
> To the patches:
>
> 4e15ee0e24 Don't hardcode tmp_check/ as test directory for tap tests
> 1a3169bc3f Split TESTDIR into TESTLOGDIR and TESTDATADIR
>
> It's a bit weird that the first patch changes the meaning of TESTDIR
> and the second patch removes it.  Maybe these patches should be
> squashed together?

Hm, to me they seem topically separate enough, but I don't have a strong
opinion on it.


> 581721fa99 meson: prereq: Add src/tools/gen_export.pl
>
> Still wondering about the whitespace changes I reported recently, but
> that can also be fine-tuned later.

I'll look into it in a bit.


> a1fb97a81b meson: Add meson based buildsystem
>
> I'm not a fan of all this business to protect the two build systems
> from each other.  I don't like the build process touching a file under
> version control every time.  How necessary is this?  What happens
> otherwise?

I added it after just about everyone trying meson hit problems due to
conflicts between (past) in-tree configure builds and meson, due to files left
in tree (picking up the wrong .h files, cannot entirely be fixed with -I
arguments, due to the "" includes). By adding the relevant check to the meson
configure phase, and by triggering meson re-configure whenever an in-tree
configure build is done, these issues can be detected.

It'd of course be nicer to avoid the potential for such conflicts, but that
appears to be a huge chunk of work, see the bison/flex subthread.

So I don't really see an alternative.


> conversion_helpers.txt: should probably be removed now.

Done.


> doc/src/sgml/resolv.xsl: I don't understand what this is doing.  Maybe
> at least add a comment in the file.

It's only used for building epubs. Perhaps I should extract that into a
separate patch as well? The relevant section is:

> #
> # epub
> #
>
> # This was previously implemented using dbtoepub - but that doesn't seem to
> # support running in build != source directory (i.e. VPATH builds already
> # weren't supported).
> if pandoc.found() and xsltproc.found()
>   # XXX: Wasn't able to make pandoc successfully resolve entities
>   # XXX: Perhaps we should just make all targets use this, to avoid repeatedly
>   # building whole thing? It's comparatively fast though.
>   postgres_full_xml = custom_target('postgres-full.xml',
>     input: ['resolv.xsl', 'postgres.sgml'],
>     output: ['postgres-full.xml'],
>     depends: doc_generated + [postgres_sgml_valid],
>     command: [xsltproc, '--path', '@OUTDIR@/', xsltproc_flags,
>               '-o', '@OUTPUT@', '@INPUT@'],
>     build_by_default: false,
>   )

A noted, I couldn't make pandoc resolve our entities, so I used resolv.xsl
them, before calling pandoc.

I'll rename it to resolve-entities.xsl and add a comment.


> src/common/unicode/meson.build: The comment at the top of the file
> should be moved next to the files it is describing (similar to how it
> is in the makefile).

Done.


> I don't see CLDR_VERSION set anywhere.  Is that part implemented?

No, I didn't implement the generation parts of contrib/unaccent. I started
tackling the src/common/unicode bits after John Naylor asked whether that
could be done, but considered that good enough...


> src/port/win32ver.rc.in: This is redundant with src/port/win32ver.rc.
> (Note that the latter is also used as an input file for text
> substitution.  So having another file named *.in next to it would be
> super confusing.)

Yea, this stuff isn't great. I think the better solution, both for meson and
for configure, would be to move to do all the substitution to the C
preprocessor.


> src/tools/find_meson: Could use a brief comment what it does.

Added.


> src/tools/pgflex: Could use a not-brief comment about what it does,
> why it's needed.  Also a comment where it's used.  Also run this
> through pycodestyle.

Working on that.


> cd193eb3e8 meson: ci: Build both with meson and as before
>
> I haven't reviewed this one in detail.  Maybe add a summary in the
> commit message, like these are the new jobs, these are the changes to
> existing jobs.  It looks like there is more in there than just adding
> a few meson jobs.

I don't think we want to commit this as-is. It contains CI for a lot of
platforms - that's very useful for working on meson, but too much for
in-tree. I guess I'll split it into two, one patch for converting a reasonable
subset of the current CI tasks to meson and another to add (back) the current
array of tested platforms.


> If the above are addressed, I think this will be just about at the
> point where the above patches can be committed.

Woo!


> Everything past these patches I'm mentally postponing right now.

Makes sense.

Greetings,

Andres Freund

Commits

  1. meson: Add 'running' test setup, as a replacement for installcheck

  2. meson: Add support for building with precompiled headers

  3. windows: Adjust FD_SETSIZE via commandline define

  4. meson: docs: Add xml{lint,proc} wrapper to collect dependencies

  5. meson: ecpg: Split definition of static and shared libraries

  6. windows: Set UMDF_USING_NTSTATUS globally, include ntstatus.h

  7. ci: Add hint about downloadable logs to README

  8. meson: Set up absolute rpaths to libdir

  9. meson: Include CFLAGS/c_args in summary and pg_config output

  10. windows: remove date from version number in win32ver.rc

  11. meson: Add initial version of meson based build system

  12. ci: windows: set error mode to not include SEM_NOGPFAULTERRORBOX

  13. Refactor PG_TEST_EXTRA logic in autoconf build

  14. Split TESTDIR into TESTLOGDIR and TESTDATADIR

  15. Don't hardcode tmp_check/ as test directory for tap tests

  16. Extend gendef.pl in preparation for meson

  17. Include c.h instead of postgres.h in src/port/*p{read,write}*.c

  18. Remove DLLTOOL, DLLWRAP from configure / Makefile.global.in

  19. Run xmllint validation only once

  20. Bump minimum Perl version to 5.14

  21. Move gramparse.h to src/backend/parser

  22. Run perltidy over Catalog.pm

  23. Parse catalog .dat files as a whole when compiling the backend

  24. Build all Flex files standalone

  25. Move private declarations shared between guc.c and guc-file.l to new header

  26. Preparatory refactoring for compiling guc-file.c standalone

  27. Move darwin sysroot determination into separate file

  28. Fix MSVC warning in compat_informix/rnull.pgc

  29. solaris: Use versioning scripts instead of -Bsymbolic

  30. Change shared library installation naming on macOS

  31. regress: allow to specify directory containing expected files, for ecpg

  32. Don't add HAVE_LDAP_H HAVE_WINLDAP_H to pg_config.h

  33. Refactor dtrace postprocessing make rules

  34. Add output directory option to gen_node_support.pl

  35. Add output directory argument to generate-unicode_norm_table.pl

  36. Add output file argument to generate-errcodes.pl

  37. Add output path arg in generate-lwlocknames.pl

  38. Move snowball_create.sql creation into perl file

  39. ecpg: Output dir, source dir, stamp file argument for preproc/*.pl

  40. psql: Output dir and dependency generation for sql_help

  41. Deal with paths containing \ and spaces in basebackup_to_shell tests

  42. Remove LLVM_CONFIG from Makefile.global.in

  43. Make update-unicode target work in vpath builds

  44. Refactor DLSUFFIX handling

  45. ldap tests: Add paths for openbsd.

  46. ldap tests: Don't run on unsupported operating systems.

  47. Remove check for accept() argument types

  48. Move our p{read,write}v replacements into their own files.

  49. Adjust yywrap macro for non-reentrant scanners for MSVC.

  50. Remove any -arch switches given in ExtUtils::Embed's ldopts from our

  51. Change PL/Perl and Pg interface build to use configured compiler and

  52. Apparently, on some systems, ExtUtils::Embed and MakeMaker are slightly