Horton

Thread

Commits

  1. windows: Adjust FD_SETSIZE via commandline define

  1. meson: Add support for building with precompiled headers

    Andres Freund <andres@anarazel.de> — 2022-10-05T19:08:29Z

    Hi,

This is a patch split off from the initial meson thread [1] as it's
functionally largely independent (as suggested in [2]).

Using precompiled headers substantially speeds up building for windows, due to
the vast amount of headers included via windows.h. A cross build from
linux targetting mingw goes from

994.11user 136.43system 0:31.58elapsed 3579%CPU
to
422.41user 89.05system 0:14.35elapsed 3562%CPU

The wins on windows are similar-ish (but I don't have a system at hand just
now for actual numbers). Targetting other operating systems the wins are far
smaller (tested linux, macOS, FreeBSD).

This is particularly interesting for cfbot, which spends a lot of time
building on windows.  It also makes developing on windows less painful as the
gains are bigger when compiling incrementally, because the precompiled headers
don't typically have to be rebuilt.


As a prerequisite this requires changing the way FD_SETSIZE is defined when
targetting windows.

When using precompiled headers we cannot override macros in system headers
from within .c files, as headers are already processed before the #define in
the C file is reached.

A few files #define FD_SETSIZE 1024 on windows, as the default is only 64. I
am hesitant to change FD_SETSIZE globally on windows, due to
src/backend/port/win32/socket.c using it to size on-stack arrays. Instead add
-DFD_SETSIZE=1024 when building the specific targets needing it.

We likely should move away from using select() in those places, but that's a
larger change.


Michael, CCing you wrt the second patch, as Thomas noticed [3] that you were
looking at where to define FD_SETSIZE.

Greetings,

Andres Freund

[1] https://www.postgresql.org/message-id/20211012083721.hvixq4pnh2pixr3j%40alap3.anarazel.de
[2] https://www.postgresql.org/message-id/e0c44fb2-8b66-a4b9-b274-7ed3a1a0ab74%40enterprisedb.com
[3] https://www.postgresql.org/message-id/CA+hUKG+50eOUbN++ocDc0Qnp9Pvmou23DSXu=ZA6fepOcftKqA@mail.gmail.com
[4] https://www.postgresql.org/message-id/20190826054000.GE7005%40paquier.xyz

  2. Re: meson: Add support for building with precompiled headers

    Tom Lane <tgl@sss.pgh.pa.us> — 2022-10-05T20:09:14Z

    Andres Freund <andres@anarazel.de> writes:
> When using precompiled headers we cannot override macros in system headers
> from within .c files, as headers are already processed before the #define in
> the C file is reached.

> A few files #define FD_SETSIZE 1024 on windows, as the default is only 64. I
> am hesitant to change FD_SETSIZE globally on windows, due to
> src/backend/port/win32/socket.c using it to size on-stack arrays. Instead add
> -DFD_SETSIZE=1024 when building the specific targets needing it.

Color me confused, but how does it work to #define that from the command
line if it can't be overridden from within the program?

			regards, tom lane

  3. Re: meson: Add support for building with precompiled headers

    Andres Freund <andres@anarazel.de> — 2022-10-05T20:17:42Z

    Hi,

On 2022-10-05 16:09:14 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > When using precompiled headers we cannot override macros in system headers
> > from within .c files, as headers are already processed before the #define in
> > the C file is reached.
> 
> > A few files #define FD_SETSIZE 1024 on windows, as the default is only 64. I
> > am hesitant to change FD_SETSIZE globally on windows, due to
> > src/backend/port/win32/socket.c using it to size on-stack arrays. Instead add
> > -DFD_SETSIZE=1024 when building the specific targets needing it.
> 
> Color me confused, but how does it work to #define that from the command
> line if it can't be overridden from within the program?

If specified on the commandline it's also used when generating the precompiled
header - of course that's not possible when it's just #define'd in some .c
file.

Greetings,

Andres Freund

  4. Re: meson: Add support for building with precompiled headers

    Tom Lane <tgl@sss.pgh.pa.us> — 2022-10-05T20:21:55Z

    Andres Freund <andres@anarazel.de> writes:
> On 2022-10-05 16:09:14 -0400, Tom Lane wrote:
>> Color me confused, but how does it work to #define that from the command
>> line if it can't be overridden from within the program?

> If specified on the commandline it's also used when generating the precompiled
> header - of course that's not possible when it's just #define'd in some .c
> file.

Ah, so there's a separate cache of precompiled headers for each set of
compiler command-line arguments?  Got it.

			regards, tom lane

  5. Re: meson: Add support for building with precompiled headers

    Andres Freund <andres@anarazel.de> — 2022-10-05T20:27:16Z

    Hi,

On 2022-10-05 16:21:55 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2022-10-05 16:09:14 -0400, Tom Lane wrote:
> >> Color me confused, but how does it work to #define that from the command
> >> line if it can't be overridden from within the program?
> 
> > If specified on the commandline it's also used when generating the precompiled
> > header - of course that's not possible when it's just #define'd in some .c
> > file.
> 
> Ah, so there's a separate cache of precompiled headers for each set of
> compiler command-line arguments?  Got it.

Worse, it builds the precompiled header for each "target" (static/shared lib,
executable), right now. Hence I've only added them for targets that have
multiple .c files. I've been planning to submit an improvement to meson that
does what you propose, it'd not be hard, but before it's actually usable, it
didn't seem worth investing time in that.

Greetings,

Andres Freund

  6. Re: meson: Add support for building with precompiled headers

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2022-10-06T07:06:42Z

    On 05.10.22 21:08, Andres Freund wrote:
> This is a patch split off from the initial meson thread [1] as it's
> functionally largely independent (as suggested in [2]).
> 
> Using precompiled headers substantially speeds up building for windows, due to
> the vast amount of headers included via windows.h. A cross build from
> linux targetting mingw goes from

These patches look ok to me.  I can't really comment on the Windows 
details, but it sounds all reasonable.

Small issue:

+override CFLAGS += -DFD_SETSIZE=1024

(and similar)

should be CPPFLAGS.

  7. Re: meson: Add support for building with precompiled headers

    Andres Freund <andres@anarazel.de> — 2022-10-07T00:25:24Z

    Hi,

On 2022-10-06 09:06:42 +0200, Peter Eisentraut wrote:
> On 05.10.22 21:08, Andres Freund wrote:
> > This is a patch split off from the initial meson thread [1] as it's
> > functionally largely independent (as suggested in [2]).
> > 
> > Using precompiled headers substantially speeds up building for windows, due to
> > the vast amount of headers included via windows.h. A cross build from
> > linux targetting mingw goes from
> 
> These patches look ok to me.  I can't really comment on the Windows details,
> but it sounds all reasonable.

Thanks for reviewing!


> Small issue:
> 
> +override CFLAGS += -DFD_SETSIZE=1024
> 
> (and similar)
> 
> should be CPPFLAGS.

Pushed with that adjusted.

Greetings,

Andres Freund