Thread
Commits
-
Avoid unnecessary "touch meson.build" in vpath builds
- 6d4bc96eb77b 17.0 landed
- 2209c70c652e 16.3 landed
-
meson: Add initial version of meson based build system
- e6927270cd18 16.0 cited
-
pgsql: meson: Add initial version of meson based build system
Andres Freund <andres@anarazel.de> — 2022-09-22T05:47:52Z
meson: Add initial version of meson based build system Autoconf is showing its age, fewer and fewer contributors know how to wrangle it. Recursive make has a lot of hard to resolve dependency issues and slow incremental rebuilds. Our home-grown MSVC build system is hard to maintain for developers not using Windows and runs tests serially. While these and other issues could individually be addressed with incremental improvements, together they seem best addressed by moving to a more modern build system. After evaluating different build system choices, we chose to use meson, to a good degree based on the adoption by other open source projects. We decided that it's more realistic to commit a relatively early version of the new build system and mature it in tree. This commit adds an initial version of a meson based build system. It supports building postgres on at least AIX, FreeBSD, Linux, macOS, NetBSD, OpenBSD, Solaris and Windows (however only gcc is supported on aix, solaris). For Windows/MSVC postgres can now be built with ninja (faster, particularly for incremental builds) and msbuild (supporting the visual studio GUI, but building slower). Several aspects (e.g. Windows rc file generation, PGXS compatibility, LLVM bitcode generation, documentation adjustments) are done in subsequent commits requiring further review. Other aspects (e.g. not installing test-only extensions) are not yet addressed. When building on Windows with msbuild, builds are slower when using a visual studio version older than 2019, because those versions do not support MultiToolTask, required by meson for intra-target parallelism. The plan is to remove the MSVC specific build system in src/tools/msvc soon after reaching feature parity. However, we're not planning to remove the autoconf/make build system in the near future. Likely we're going to keep at least the parts required for PGXS to keep working around until all supported versions build with meson. Some initial help for postgres developers is at https://wiki.postgresql.org/wiki/Meson With contributions from Thomas Munro, John Naylor, Stone Tickle and others. Author: Andres Freund <andres@anarazel.de> Author: Nazir Bilal Yavuz <byavuz81@gmail.com> Author: Peter Eisentraut <peter@eisentraut.org> Reviewed-By: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://postgr.es/m/20211012083721.hvixq4pnh2pixr3j@alap3.anarazel.de Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/e6927270cd18d535b77cbe79c55c6584351524be Modified Files -------------- config/meson.build | 4 + configure | 6 + configure.ac | 6 + contrib/adminpack/meson.build | 23 + contrib/amcheck/meson.build | 37 + contrib/auth_delay/meson.build | 5 + contrib/auto_explain/meson.build | 16 + contrib/basebackup_to_shell/meson.build | 22 + contrib/basic_archive/meson.build | 23 + contrib/bloom/meson.build | 36 + contrib/bool_plperl/meson.build | 42 + contrib/btree_gin/meson.build | 54 + contrib/btree_gist/meson.build | 84 + contrib/citext/meson.build | 34 + contrib/cube/meson.build | 53 + contrib/dblink/meson.build | 31 + contrib/dict_int/meson.build | 22 + contrib/dict_xsyn/meson.build | 29 + contrib/earthdistance/meson.build | 23 + contrib/file_fdw/meson.build | 22 + contrib/fuzzystrmatch/meson.build | 26 + contrib/hstore/meson.build | 44 + contrib/hstore_plperl/meson.build | 43 + contrib/hstore_plpython/meson.build | 37 + contrib/intagg/meson.build | 6 + contrib/intarray/meson.build | 37 + contrib/isn/meson.build | 33 + contrib/jsonb_plperl/meson.build | 43 + contrib/jsonb_plpython/meson.build | 36 + contrib/lo/meson.build | 27 + contrib/ltree/meson.build | 44 + contrib/ltree_plpython/meson.build | 37 + contrib/meson.build | 66 + contrib/oid2name/meson.build | 17 + contrib/old_snapshot/meson.build | 15 + contrib/pageinspect/meson.build | 50 + contrib/passwordcheck/meson.build | 30 + contrib/pg_buffercache/meson.build | 27 + contrib/pg_freespacemap/meson.build | 29 + contrib/pg_prewarm/meson.build | 27 + contrib/pg_stat_statements/meson.build | 35 + contrib/pg_surgery/meson.build | 25 + contrib/pg_trgm/meson.build | 35 + contrib/pg_visibility/meson.build | 26 + contrib/pg_walinspect/meson.build | 27 + contrib/pgcrypto/meson.build | 100 + contrib/pgrowlocks/meson.build | 27 + contrib/pgstattuple/meson.build | 31 + contrib/postgres_fdw/meson.build | 34 + contrib/seg/meson.build | 51 + contrib/sepgsql/meson.build | 34 + contrib/spi/meson.build | 50 + contrib/sslinfo/meson.build | 21 + contrib/tablefunc/meson.build | 24 + contrib/tcn/meson.build | 25 + contrib/test_decoding/meson.build | 63 + contrib/tsm_system_rows/meson.build | 24 + contrib/tsm_system_time/meson.build | 24 + contrib/unaccent/meson.build | 32 + contrib/uuid-ossp/meson.build | 31 + contrib/vacuumlo/meson.build | 17 + contrib/xml2/meson.build | 32 + doc/src/sgml/meson.build | 254 ++ doc/src/sgml/version.sgml.in | 2 + meson.build | 3025 ++++++++++++++++++++ meson_options.txt | 185 ++ src/backend/access/brin/meson.build | 12 + src/backend/access/common/meson.build | 18 + src/backend/access/gin/meson.build | 17 + src/backend/access/gist/meson.build | 13 + src/backend/access/hash/meson.build | 12 + src/backend/access/heap/meson.build | 11 + src/backend/access/index/meson.build | 6 + src/backend/access/meson.build | 13 + src/backend/access/nbtree/meson.build | 13 + src/backend/access/rmgrdesc/meson.build | 26 + src/backend/access/spgist/meson.build | 13 + src/backend/access/table/meson.build | 6 + src/backend/access/tablesample/meson.build | 5 + src/backend/access/transam/meson.build | 31 + src/backend/backup/meson.build | 13 + src/backend/bootstrap/meson.build | 28 + src/backend/catalog/meson.build | 44 + src/backend/commands/meson.build | 51 + src/backend/executor/meson.build | 67 + src/backend/foreign/meson.build | 3 + src/backend/jit/llvm/meson.build | 73 + src/backend/jit/meson.build | 3 + src/backend/lib/meson.build | 12 + src/backend/libpq/meson.build | 32 + src/backend/main/meson.build | 2 + src/backend/meson.build | 190 ++ src/backend/nodes/meson.build | 29 + src/backend/optimizer/geqo/meson.build | 17 + src/backend/optimizer/meson.build | 5 + src/backend/optimizer/path/meson.build | 11 + src/backend/optimizer/plan/meson.build | 10 + src/backend/optimizer/prep/meson.build | 7 + src/backend/optimizer/util/meson.build | 16 + src/backend/parser/meson.build | 48 + src/backend/partitioning/meson.build | 5 + src/backend/po/meson.build | 1 + src/backend/port/meson.build | 31 + src/backend/port/win32/meson.build | 6 + src/backend/postmaster/meson.build | 15 + src/backend/regex/meson.build | 8 + .../replication/libpqwalreceiver/meson.build | 13 + src/backend/replication/logical/meson.build | 14 + src/backend/replication/meson.build | 51 + src/backend/replication/pgoutput/meson.build | 10 + src/backend/rewrite/meson.build | 9 + src/backend/snowball/meson.build | 88 + src/backend/statistics/meson.build | 6 + src/backend/storage/buffer/meson.build | 7 + src/backend/storage/file/meson.build | 8 + src/backend/storage/freespace/meson.build | 5 + src/backend/storage/ipc/meson.build | 20 + src/backend/storage/large_object/meson.build | 3 + src/backend/storage/lmgr/meson.build | 13 + src/backend/storage/meson.build | 9 + src/backend/storage/page/meson.build | 5 + src/backend/storage/smgr/meson.build | 4 + src/backend/storage/sync/meson.build | 4 + src/backend/tcop/meson.build | 8 + src/backend/tsearch/meson.build | 21 + src/backend/utils/activity/meson.build | 18 + src/backend/utils/adt/meson.build | 131 + src/backend/utils/cache/meson.build | 16 + src/backend/utils/error/meson.build | 6 + src/backend/utils/fmgr/meson.build | 8 + src/backend/utils/hash/meson.build | 4 + src/backend/utils/init/meson.build | 4 + src/backend/utils/mb/conversion_procs/meson.build | 36 + src/backend/utils/mb/meson.build | 9 + src/backend/utils/meson.build | 17 + src/backend/utils/misc/meson.build | 35 + src/backend/utils/mmgr/meson.build | 10 + src/backend/utils/resowner/meson.build | 3 + src/backend/utils/sort/meson.build | 9 + src/backend/utils/time/meson.build | 4 + src/bin/initdb/meson.build | 30 + src/bin/initdb/po/meson.build | 1 + src/bin/meson.build | 20 + src/bin/pg_amcheck/meson.build | 27 + src/bin/pg_amcheck/po/meson.build | 1 + src/bin/pg_archivecleanup/meson.build | 19 + src/bin/pg_archivecleanup/po/meson.build | 1 + src/bin/pg_basebackup/meson.build | 61 + src/bin/pg_basebackup/po/meson.build | 1 + src/bin/pg_checksums/meson.build | 21 + src/bin/pg_checksums/po/meson.build | 1 + src/bin/pg_config/meson.build | 19 + src/bin/pg_config/po/meson.build | 1 + src/bin/pg_controldata/meson.build | 19 + src/bin/pg_controldata/po/meson.build | 1 + src/bin/pg_ctl/meson.build | 22 + src/bin/pg_ctl/po/meson.build | 1 + src/bin/pg_dump/meson.build | 75 + src/bin/pg_dump/po/meson.build | 1 + src/bin/pg_resetwal/meson.build | 20 + src/bin/pg_resetwal/po/meson.build | 1 + src/bin/pg_rewind/meson.build | 42 + src/bin/pg_rewind/po/meson.build | 1 + src/bin/pg_test_fsync/meson.build | 21 + src/bin/pg_test_fsync/po/meson.build | 1 + src/bin/pg_test_timing/meson.build | 19 + src/bin/pg_test_timing/po/meson.build | 1 + src/bin/pg_upgrade/meson.build | 40 + src/bin/pg_upgrade/po/meson.build | 1 + src/bin/pg_verifybackup/meson.build | 33 + src/bin/pg_verifybackup/po/meson.build | 1 + src/bin/pg_waldump/meson.build | 30 + src/bin/pg_waldump/po/meson.build | 1 + src/bin/pgbench/meson.build | 38 + src/bin/pgevent/meson.build | 24 + src/bin/psql/meson.build | 67 + src/bin/psql/po/meson.build | 1 + src/bin/scripts/meson.build | 51 + src/bin/scripts/po/meson.build | 1 + src/common/meson.build | 174 ++ src/common/unicode/meson.build | 106 + src/fe_utils/meson.build | 29 + src/include/catalog/meson.build | 142 + src/include/meson.build | 173 ++ src/include/nodes/meson.build | 58 + src/include/pg_config_ext.h.meson | 7 + src/include/storage/meson.build | 19 + src/include/utils/meson.build | 57 + src/interfaces/ecpg/compatlib/meson.build | 22 + src/interfaces/ecpg/ecpglib/meson.build | 37 + src/interfaces/ecpg/ecpglib/po/meson.build | 1 + src/interfaces/ecpg/include/meson.build | 51 + src/interfaces/ecpg/meson.build | 9 + src/interfaces/ecpg/pgtypeslib/meson.build | 30 + src/interfaces/ecpg/preproc/meson.build | 104 + src/interfaces/ecpg/preproc/po/meson.build | 1 + .../ecpg/test/compat_informix/meson.build | 31 + src/interfaces/ecpg/test/compat_oracle/meson.build | 20 + src/interfaces/ecpg/test/connect/meson.build | 20 + src/interfaces/ecpg/test/meson.build | 84 + src/interfaces/ecpg/test/pgtypeslib/meson.build | 21 + src/interfaces/ecpg/test/preproc/meson.build | 37 + src/interfaces/ecpg/test/sql/meson.build | 46 + src/interfaces/ecpg/test/thread/meson.build | 21 + src/interfaces/libpq/meson.build | 108 + src/interfaces/libpq/po/meson.build | 1 + src/interfaces/libpq/test/meson.build | 15 + src/interfaces/meson.build | 2 + src/meson.build | 12 + src/pl/meson.build | 5 + src/pl/plperl/meson.build | 90 + src/pl/plperl/po/meson.build | 1 + src/pl/plpgsql/meson.build | 1 + src/pl/plpgsql/src/meson.build | 84 + src/pl/plpgsql/src/po/meson.build | 1 + src/pl/plpython/meson.build | 99 + src/pl/plpython/po/meson.build | 1 + src/pl/tcl/meson.build | 55 + src/pl/tcl/po/meson.build | 1 + src/port/meson.build | 184 ++ src/test/authentication/meson.build | 11 + src/test/icu/meson.build | 11 + src/test/isolation/meson.build | 58 + src/test/kerberos/meson.build | 15 + src/test/ldap/meson.build | 11 + src/test/meson.build | 25 + src/test/modules/brin/meson.build | 16 + src/test/modules/commit_ts/meson.build | 18 + src/test/modules/delay_execution/meson.build | 18 + src/test/modules/dummy_index_am/meson.build | 23 + src/test/modules/dummy_seclabel/meson.build | 23 + src/test/modules/libpq_pipeline/meson.build | 21 + src/test/modules/meson.build | 27 + src/test/modules/plsample/meson.build | 23 + src/test/modules/snapshot_too_old/meson.build | 14 + src/test/modules/spgist_name_ops/meson.build | 23 + .../modules/ssl_passphrase_callback/meson.build | 48 + src/test/modules/test_bloomfilter/meson.build | 23 + src/test/modules/test_ddl_deparse/meson.build | 43 + src/test/modules/test_extensions/meson.build | 45 + src/test/modules/test_ginpostinglist/meson.build | 23 + src/test/modules/test_integerset/meson.build | 23 + src/test/modules/test_lfind/meson.build | 23 + src/test/modules/test_misc/meson.build | 12 + src/test/modules/test_oat_hooks/meson.build | 18 + src/test/modules/test_parser/meson.build | 23 + src/test/modules/test_pg_dump/meson.build | 22 + src/test/modules/test_predtest/meson.build | 23 + src/test/modules/test_rbtree/meson.build | 23 + src/test/modules/test_regex/meson.build | 24 + src/test/modules/test_rls_hooks/meson.build | 17 + src/test/modules/test_shm_mq/meson.build | 27 + src/test/modules/unsafe_tests/meson.build | 11 + src/test/modules/worker_spi/meson.build | 26 + src/test/perl/meson.build | 12 + src/test/recovery/meson.build | 43 + src/test/regress/meson.build | 62 + src/test/ssl/meson.build | 13 + src/test/subscription/meson.build | 42 + src/timezone/meson.build | 48 + src/timezone/tznames/meson.build | 21 + src/tools/find_meson | 30 + src/tools/gen_export.pl | 81 + src/tools/pgflex | 85 + src/tools/testwrap | 47 + 265 files changed, 10962 insertions(+)
-
Re: pgsql: meson: Add initial version of meson based build system
Christoph Berg <myon@debian.org> — 2024-04-17T13:42:28Z
Re: Andres Freund > https://git.postgresql.org/pg/commitdiff/e6927270cd18d535b77cbe79c55c6584351524be This commit broke VPATH builds when the original source directory contains symlinks. The $PWD is /home/myon/postgresql/pg/master, but the actual directory is /home/myon/projects/postgresql/pg/postgresql. When I mkdir build; cd build && ../configure there, I get a build directory missing a lot of files/directories: $ ls build/ config.log config.status* GNUmakefile meson.build src/ $ ls build/src/ backend/ include/ interfaces/ Makefile.global Makefile.port@ $ ls build/src/backend/ port/ Given there are no other changes I think this bit is at fault: > Modified Files > -------------- > configure.ac | 6 + +# Ensure that any meson build directories would reconfigure and see that +# there's a conflicting in-tree build and can error out. +if test "$vpath_build"="no"; then + touch meson.build +fi Christoph
-
Re: pgsql: meson: Add initial version of meson based build system
Andres Freund <andres@anarazel.de> — 2024-04-17T23:00:02Z
Hi, Uh, huh. On 2024-04-17 15:42:28 +0200, Christoph Berg wrote: > Re: Andres Freund > > https://git.postgresql.org/pg/commitdiff/e6927270cd18d535b77cbe79c55c6584351524be > > This commit broke VPATH builds when the original source directory > contains symlinks. I.e. a symlink to the source directory, not a symlink inside the source directory. > Given there are no other changes I think this bit is at fault: > > > Modified Files > > -------------- > > configure.ac | 6 + > > +# Ensure that any meson build directories would reconfigure and see that > +# there's a conflicting in-tree build and can error out. > +if test "$vpath_build"="no"; then > + touch meson.build > +fi Argh, this is missing spaces around the '=', leading to the branch always being entered. What I don't understand is how that possibly could affect the prep_buildtree step, that happens earlier. Hm. Uh, I don't think it does? Afaict this failure is entirely unrelated to 'touch meson.build'? From what I can tell the problem is that config/prep_buildtree is invoked with the symlinked path, and that that doesn't seem to work: bash -x /home/andres/src/postgresql-via-symlink/config/prep_buildtree /home/andres/src/postgresql-via-symlink . ++ basename /home/andres/src/postgresql-via-symlink/config/prep_buildtree + me=prep_buildtree + help='Usage: prep_buildtree sourcetree [buildtree]' + test -z /home/andres/src/postgresql-via-symlink + test x/home/andres/src/postgresql-via-symlink = x--help + unset CDPATH ++ cd /home/andres/src/postgresql-via-symlink ++ pwd + sourcetree=/home/andres/src/postgresql-via-symlink ++ cd . ++ pwd + buildtree=/tmp/pgs ++ find /home/andres/src/postgresql-via-symlink -type d '(' '(' -name CVS -prune ')' -o '(' -name .git -prune ')' -o -print ')' ++ grep -v '/home/andres/src/postgresql-via-symlink/doc/src/sgml/\+' ++ find /home/andres/src/postgresql-via-symlink -name Makefile -print -o -name GNUmakefile -print ++ grep -v /home/andres/src/postgresql-via-symlink/doc/src/sgml/images/ + exit 0 Note that the find does not return anything. Greetings, Andres Freund -
Re: pgsql: meson: Add initial version of meson based build system
Christoph Berg <myon@debian.org> — 2024-04-18T08:54:18Z
Re: Andres Freund > > This commit broke VPATH builds when the original source directory > > contains symlinks. > > I.e. a symlink to the source directory, not a symlink inside the source > directory. Yes. > Argh, this is missing spaces around the '=', leading to the branch always > being entered. Glad I found at least something :) > Uh, I don't think it does? Afaict this failure is entirely unrelated to 'touch > meson.build'? From what I can tell the problem is that config/prep_buildtree > is invoked with the symlinked path, and that that doesn't seem to work: Apparently I messed up both the git bisect run and manually confirm the problem later. Trying again now, the problem has been existing at least since 2002, probably earlier. I've been using this directory layout for years, apparently so far I've always only used non-VPATH builds or dpkg-buildpackage, which probably canonicalizes the path before building, given it works. Since no one else has been complaining, it might not be worth fixing. Sorry for the noise! Christoph
-
Re: pgsql: meson: Add initial version of meson based build system
Andres Freund <andres@anarazel.de> — 2024-04-25T14:58:22Z
Hi, On 2024-04-18 10:54:18 +0200, Christoph Berg wrote: > Re: Andres Freund > > > This commit broke VPATH builds when the original source directory > > > contains symlinks. > > Argh, this is missing spaces around the '=', leading to the branch always > > being entered. > > Glad I found at least something :) Yep :). I pushed a fix to that now. > I've been using this directory layout for years, apparently so far > I've always only used non-VPATH builds or dpkg-buildpackage, which > probably canonicalizes the path before building, given it works. I wonder if perhaps find's behaviour might have changed at some point? > Since no one else has been complaining, it might not be worth fixing. I'm personally not excited about working on fixing this, but if somebody else wants to spend the cycles to make this work reliably... It's certainly interesting that we have some code worrying about symlinks in configure.ac: # prepare build tree if outside source tree # Note 1: test -ef might not exist, but it's more reliable than `pwd`. # Note 2: /bin/pwd might be better than shell's built-in at getting # a symlink-free name. But we only use this to determine if we're doing a vpath build, not as the path passed to prep_buildtree... Greetings, Andres Freund