Horton

Re: [RFC] building postgres with meson - v12

John Naylor <john.naylor@enterprisedb.com>

View thread View on postgr.es
From: John Naylor <john.naylor@enterprisedb.com>
To: Andres Freund <andres@anarazel.de>
Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, samay sharma <smilingsamay@gmail.com>, Nazir Bilal Yavuz <byavuz81@gmail.com>
Date: 2022-09-04T06:12:52Z
Lists: pgsql-hackers

Attachments

On Fri, Sep 2, 2022 at 11:35 PM Andres Freund <andres@anarazel.de> wrote:
>
> Hi,
>
> On 2022-09-02 14:17:26 +0700, John Naylor wrote:
> > On Thu, Sep 1, 2022 at 1:12 AM Andres Freund <andres@anarazel.de> wrote:
> > > [v12]
> >
> > +# Build a small utility static lib for the parser. This makes it easier to not
> > +# depend on gram.h already having been generated for most of the other code
> > +# (which depends on generated headers having been generated). The generation
> > +# of the parser is slow...
> >
> > It's not obvious whether this is intended to be a Meson-only
> > optimization or a workaround for something awkward to specify.
>
> It is an optimization. The parser generation is by far the slowest part of a
> build. If other files can only be compiled once gram.h is generated, there's a
> long initial period where little can happen. So instead of having all .c files
> have a dependency on gram.h having been generated, the above makes only
> scan.c, gram.c compilation depend on gram.h.  It only matters for the first
> compilation, because such dependencies are added as order-only dependencies,
> supplanted by more precise compiler generated dependencies after.

Okay, I think the comment could include some of this info for clarity.

> It's still pretty annoying that so much of the build is initially idle,
> waiting for genbki.pl to finish.
>
> Part of that is due to some ugly dependencies of src/common on backend headers
> that IMO probably shouldn't exist (e.g. src/common/relpath.c includes
> catalog/pg_tablespace_d.h).

Technically, *_d.h headers are not backend, that's why it's safe to
include them anywhere. relpath.c in its current form has to know the
tablespace OIDs, which I guess is what you think is ugly. (I agree
it's not great)

> Looks like it'd not be hard to get at least the
> _shlib version of src/common and libpq build without waiting for that. But for
> all the backend code I don't really see a way, so it'd be nice to make genbki
> faster at some point.

The attached gets me a ~15% reduction in clock time by having
Catalog.pm parse the .dat files in one sweep, when we don't care about
formatting, i.e. most of the time:

master:
User time (seconds): 0.48
Maximum resident set size (kbytes): 36112

patch:
User time (seconds): 0.41
Maximum resident set size (kbytes): 35808

That's pretty simple -- I think going beyond that would require some
perl profiling.

-- 
John Naylor
EDB: http://www.enterprisedb.com

Commits

  1. meson: Add 'running' test setup, as a replacement for installcheck

  2. meson: Add support for building with precompiled headers

  3. windows: Adjust FD_SETSIZE via commandline define

  4. meson: docs: Add xml{lint,proc} wrapper to collect dependencies

  5. meson: ecpg: Split definition of static and shared libraries

  6. windows: Set UMDF_USING_NTSTATUS globally, include ntstatus.h

  7. ci: Add hint about downloadable logs to README

  8. meson: Set up absolute rpaths to libdir

  9. meson: Include CFLAGS/c_args in summary and pg_config output

  10. windows: remove date from version number in win32ver.rc

  11. meson: Add initial version of meson based build system

  12. ci: windows: set error mode to not include SEM_NOGPFAULTERRORBOX

  13. Refactor PG_TEST_EXTRA logic in autoconf build

  14. Split TESTDIR into TESTLOGDIR and TESTDATADIR

  15. Don't hardcode tmp_check/ as test directory for tap tests

  16. Extend gendef.pl in preparation for meson

  17. Include c.h instead of postgres.h in src/port/*p{read,write}*.c

  18. Remove DLLTOOL, DLLWRAP from configure / Makefile.global.in

  19. Run xmllint validation only once

  20. Bump minimum Perl version to 5.14

  21. Move gramparse.h to src/backend/parser

  22. Run perltidy over Catalog.pm

  23. Parse catalog .dat files as a whole when compiling the backend

  24. Build all Flex files standalone

  25. Move private declarations shared between guc.c and guc-file.l to new header

  26. Preparatory refactoring for compiling guc-file.c standalone

  27. Move darwin sysroot determination into separate file

  28. Fix MSVC warning in compat_informix/rnull.pgc

  29. solaris: Use versioning scripts instead of -Bsymbolic

  30. Change shared library installation naming on macOS

  31. regress: allow to specify directory containing expected files, for ecpg

  32. Don't add HAVE_LDAP_H HAVE_WINLDAP_H to pg_config.h

  33. Refactor dtrace postprocessing make rules

  34. Add output directory option to gen_node_support.pl

  35. Add output directory argument to generate-unicode_norm_table.pl

  36. Add output file argument to generate-errcodes.pl

  37. Add output path arg in generate-lwlocknames.pl

  38. Move snowball_create.sql creation into perl file

  39. ecpg: Output dir, source dir, stamp file argument for preproc/*.pl

  40. psql: Output dir and dependency generation for sql_help

  41. Deal with paths containing \ and spaces in basebackup_to_shell tests

  42. Remove LLVM_CONFIG from Makefile.global.in

  43. Make update-unicode target work in vpath builds

  44. Refactor DLSUFFIX handling

  45. ldap tests: Add paths for openbsd.

  46. ldap tests: Don't run on unsupported operating systems.

  47. Remove check for accept() argument types

  48. Move our p{read,write}v replacements into their own files.

  49. Adjust yywrap macro for non-reentrant scanners for MSVC.

  50. Remove any -arch switches given in ExtUtils::Embed's ldopts from our

  51. Change PL/Perl and Pg interface build to use configured compiler and

  52. Apparently, on some systems, ExtUtils::Embed and MakeMaker are slightly