Thread

1. zlib detection in Meson on Windows broken?

   Dave Page <dpage@pgadmin.org> — 2024-05-20T10:58:05Z

   Hi
   
   I'm working on updating the build of PostgreSQL that pgAdmin uses in its
   Windows installers to use Meson ready for the v17 release. I'm using Visual
   Studio 2022, on Windows Server 2022.
   
   I've been unable to persuade Meson to detect zlib, whilst OpenSSL seems to
   be fine.
   
   The dependencies have been built and installed as follows:
   
    mkdir c:\build64
   
    wget https://zlib.net/zlib-1.3.2.tar.gz
    tar -zxvf zlib-1.3.2.tar.gz
    cd zlib-1.3.2
    cmake -DCMAKE_INSTALL_PREFIX=C:/build64/zlib -G "Visual Studio 17 2022" .
    msbuild ALL_BUILD.vcxproj /p:Configuration=Release
    msbuild RUN_TESTS.vcxproj /p:Configuration=Release
    msbuild INSTALL.vcxproj /p:Configuration=Release
    cd ..
   
    wget https://www.openssl.org/source/openssl-3.0.13.tar.gz
    tar -zxvf openssl-3.0.13.tar.gz
    cd openssl-3.0.013
    perl Configure VC-WIN64A no-asm --prefix=C:\build64\openssl no-ssl3 no-comp
    nmake
    nmake test
    nmake install
    cd ..
   
   This results in the following headers and libraries being installed for
   zlib:
   
   C:\Users\dpage\git\postgresql>dir C:\build64\zlib\include
    Volume in drive C has no label.
    Volume Serial Number is 3AAD-5864
   
    Directory of C:\build64\zlib\include
   
   17/05/2024  15:56    <DIR>          .
   17/05/2024  15:56    <DIR>          ..
   17/05/2024  15:54            17,096 zconf.h
   22/01/2024  19:32            96,829 zlib.h
                  2 File(s)        113,925 bytes
                  2 Dir(s)  98,842,726,400 bytes free
   
   C:\Users\dpage\git\postgresql>dir C:\build64\zlib\lib
    Volume in drive C has no label.
    Volume Serial Number is 3AAD-5864
   
    Directory of C:\build64\zlib\lib
   
   17/05/2024  17:01    <DIR>          .
   17/05/2024  15:56    <DIR>          ..
   17/05/2024  15:55            16,638 zlib.lib
   17/05/2024  15:55           184,458 zlibstatic.lib
                  2 File(s)        201,096 bytes
                  2 Dir(s)  98,842,726,400 bytes free
   
   I then attempt to build PostgreSQL:
   
    meson setup build
   -Dextra_include_dirs=C:/build64/openssl/include,C:/build64/zlib/include
   -Dextra_lib_dirs=C:/build64/openssl/lib,C:/build64/zlib/lib -Dssl=openssl
   -Dzlib=enabled --prefix=c:/build64/pgsql
   
   Which results in the output in output.txt, indicating that OpenSSL was
   correctly found, but zlib was not. I've also attached the meson log.
   
   I have very little experience with Meson, and even less interpreting it's
   logs, but it seems to me that it's not including the extra lib and include
   directories when it runs the test compile, given the command line it's
   reporting:
   
   cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c
   /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
   
   Bug, or am I doing something silly?
   
   
   -- 
   Dave Page
   pgAdmin: https://www.pgadmin.org
   PostgreSQL: https://www.postgresql.org
   EDB: https://www.enterprisedb.com
   

2. Re: zlib detection in Meson on Windows broken?

   Andrew Dunstan <andrew@dunslane.net> — 2024-05-20T15:52:24Z

   On 2024-05-20 Mo 06:58, Dave Page wrote:
   > Hi
   >
   > I'm working on updating the build of PostgreSQL that pgAdmin uses in 
   > its Windows installers to use Meson ready for the v17 release. I'm 
   > using Visual Studio 2022, on Windows Server 2022.
   >
   > I've been unable to persuade Meson to detect zlib, whilst OpenSSL 
   > seems to be fine.
   >
   > The dependencies have been built and installed as follows:
   >
   >  mkdir c:\build64
   >
   >  wget https://zlib.net/zlib-1.3.2.tar.gz
   >  tar -zxvf zlib-1.3.2.tar.gz
   >  cd zlib-1.3.2
   >  cmake -DCMAKE_INSTALL_PREFIX=C:/build64/zlib -G "Visual Studio 17 2022" .
   >  msbuild ALL_BUILD.vcxproj /p:Configuration=Release
   >  msbuild RUN_TESTS.vcxproj /p:Configuration=Release
   >  msbuild INSTALL.vcxproj /p:Configuration=Release
   >  cd ..
   >
   >  wget https://www.openssl.org/source/openssl-3.0.13.tar.gz
   >  tar -zxvf openssl-3.0.13.tar.gz
   >  cd openssl-3.0.013
   >  perl Configure VC-WIN64A no-asm --prefix=C:\build64\openssl no-ssl3 
   > no-comp
   >  nmake
   >  nmake test
   >  nmake install
   >  cd ..
   >
   > This results in the following headers and libraries being installed 
   > for zlib:
   >
   > C:\Users\dpage\git\postgresql>dir C:\build64\zlib\include
   >  Volume in drive C has no label.
   >  Volume Serial Number is 3AAD-5864
   >
   >  Directory of C:\build64\zlib\include
   >
   > 17/05/2024  15:56    <DIR>          .
   > 17/05/2024  15:56    <DIR>          ..
   > 17/05/2024  15:54            17,096 zconf.h
   > 22/01/2024  19:32            96,829 zlib.h
   >                2 File(s)        113,925 bytes
   >                2 Dir(s)  98,842,726,400 bytes free
   >
   > C:\Users\dpage\git\postgresql>dir C:\build64\zlib\lib
   >  Volume in drive C has no label.
   >  Volume Serial Number is 3AAD-5864
   >
   >  Directory of C:\build64\zlib\lib
   >
   > 17/05/2024  17:01    <DIR>          .
   > 17/05/2024  15:56    <DIR>          ..
   > 17/05/2024  15:55            16,638 zlib.lib
   > 17/05/2024  15:55           184,458 zlibstatic.lib
   >                2 File(s)        201,096 bytes
   >                2 Dir(s)  98,842,726,400 bytes free
   >
   > I then attempt to build PostgreSQL:
   >
   >  meson setup build 
   > -Dextra_include_dirs=C:/build64/openssl/include,C:/build64/zlib/include 
   > -Dextra_lib_dirs=C:/build64/openssl/lib,C:/build64/zlib/lib 
   > -Dssl=openssl -Dzlib=enabled --prefix=c:/build64/pgsql
   >
   > Which results in the output in output.txt, indicating that OpenSSL was 
   > correctly found, but zlib was not. I've also attached the meson log.
   >
   > I have very little experience with Meson, and even less interpreting 
   > it's logs, but it seems to me that it's not including the extra lib 
   > and include directories when it runs the test compile, given the 
   > command line it's reporting:
   >
   > cl 
   > C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c 
   > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
   >
   > Bug, or am I doing something silly?
   >
   >
   >
   
   Hi Dave!
   
   
   Not sure ;-) But this works for the buildfarm animal drongo, so we 
   should be able to make it work for you. I'll contact you offlist and see 
   if I can help.
   
   
   cheers
   
   
   andrew
   
   --
   Andrew Dunstan
   EDB:https://www.enterprisedb.com
   

3. Re: zlib detection in Meson on Windows broken?

   Sandeep Thakkar <sandeep.thakkar@enterprisedb.com> — 2024-05-21T07:19:33Z

   Hi Dave,
   
   Is the .pc file generated after the successful build of zlib? If yes, then
   meson should be able to detect the installation ideally
   
   On Mon, May 20, 2024 at 4:28 PM Dave Page <dpage@pgadmin.org> wrote:
   
   > Hi
   >
   > I'm working on updating the build of PostgreSQL that pgAdmin uses in its
   > Windows installers to use Meson ready for the v17 release. I'm using Visual
   > Studio 2022, on Windows Server 2022.
   >
   > I've been unable to persuade Meson to detect zlib, whilst OpenSSL seems to
   > be fine.
   >
   > The dependencies have been built and installed as follows:
   >
   >  mkdir c:\build64
   >
   >  wget https://zlib.net/zlib-1.3.2.tar.gz
   >  tar -zxvf zlib-1.3.2.tar.gz
   >  cd zlib-1.3.2
   >  cmake -DCMAKE_INSTALL_PREFIX=C:/build64/zlib -G "Visual Studio 17 2022" .
   >  msbuild ALL_BUILD.vcxproj /p:Configuration=Release
   >  msbuild RUN_TESTS.vcxproj /p:Configuration=Release
   >  msbuild INSTALL.vcxproj /p:Configuration=Release
   >  cd ..
   >
   >  wget https://www.openssl.org/source/openssl-3.0.13.tar.gz
   >  tar -zxvf openssl-3.0.13.tar.gz
   >  cd openssl-3.0.013
   >  perl Configure VC-WIN64A no-asm --prefix=C:\build64\openssl no-ssl3
   > no-comp
   >  nmake
   >  nmake test
   >  nmake install
   >  cd ..
   >
   > This results in the following headers and libraries being installed for
   > zlib:
   >
   > C:\Users\dpage\git\postgresql>dir C:\build64\zlib\include
   >  Volume in drive C has no label.
   >  Volume Serial Number is 3AAD-5864
   >
   >  Directory of C:\build64\zlib\include
   >
   > 17/05/2024  15:56    <DIR>          .
   > 17/05/2024  15:56    <DIR>          ..
   > 17/05/2024  15:54            17,096 zconf.h
   > 22/01/2024  19:32            96,829 zlib.h
   >                2 File(s)        113,925 bytes
   >                2 Dir(s)  98,842,726,400 bytes free
   >
   > C:\Users\dpage\git\postgresql>dir C:\build64\zlib\lib
   >  Volume in drive C has no label.
   >  Volume Serial Number is 3AAD-5864
   >
   >  Directory of C:\build64\zlib\lib
   >
   > 17/05/2024  17:01    <DIR>          .
   > 17/05/2024  15:56    <DIR>          ..
   > 17/05/2024  15:55            16,638 zlib.lib
   > 17/05/2024  15:55           184,458 zlibstatic.lib
   >                2 File(s)        201,096 bytes
   >                2 Dir(s)  98,842,726,400 bytes free
   >
   > I then attempt to build PostgreSQL:
   >
   >  meson setup build
   > -Dextra_include_dirs=C:/build64/openssl/include,C:/build64/zlib/include
   > -Dextra_lib_dirs=C:/build64/openssl/lib,C:/build64/zlib/lib -Dssl=openssl
   > -Dzlib=enabled --prefix=c:/build64/pgsql
   >
   > Which results in the output in output.txt, indicating that OpenSSL was
   > correctly found, but zlib was not. I've also attached the meson log.
   >
   > I have very little experience with Meson, and even less interpreting it's
   > logs, but it seems to me that it's not including the extra lib and include
   > directories when it runs the test compile, given the command line it's
   > reporting:
   >
   > cl
   > C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c
   > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
   >
   > Bug, or am I doing something silly?
   >
   >
   > --
   > Dave Page
   > pgAdmin: https://www.pgadmin.org
   > PostgreSQL: https://www.postgresql.org
   > EDB: https://www.enterprisedb.com
   >
   >
   
   -- 
   Sandeep Thakkar
   

4. Re: zlib detection in Meson on Windows broken?

   Nazir Bilal Yavuz <byavuz81@gmail.com> — 2024-05-21T09:14:43Z

   Hi,
   
   On Tue, 21 May 2024 at 10:20, Sandeep Thakkar
   <sandeep.thakkar@enterprisedb.com> wrote:
   >
   > Hi Dave,
   >
   > Is the .pc file generated after the successful build of zlib? If yes, then meson should be able to detect the installation ideally
   
   If meson is not able to find the .pc file automatically, using 'meson
   setup ... --pkg-config-path $ZLIB_PC_PATH' might help.
   
   -- 
   Regards,
   Nazir Bilal Yavuz
   Microsoft
   
   
   
   

5. Re: zlib detection in Meson on Windows broken?

   Dave Page <dpage@pgadmin.org> — 2024-05-21T09:41:45Z

   Hi Sandeep, Nazir,
   
   On Tue, 21 May 2024 at 10:14, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
   
   > Hi,
   >
   > On Tue, 21 May 2024 at 10:20, Sandeep Thakkar
   > <sandeep.thakkar@enterprisedb.com> wrote:
   > >
   > > Hi Dave,
   > >
   > > Is the .pc file generated after the successful build of zlib? If yes,
   > then meson should be able to detect the installation ideally
   >
   > If meson is not able to find the .pc file automatically, using 'meson
   > setup ... --pkg-config-path $ZLIB_PC_PATH' might help.
   >
   
   The problem is that on Windows there are no standard locations for a
   Unix-style development library installation such as this, so the chances
   are that the .pc file will point to entirely the wrong location.
   
   For example, please see
   https://github.com/dpage/winpgbuild/actions/runs/9172187335 which is a
   Github action that builds a completely vanilla zlib using VC++. If you look
   at the uploaded artefact containing the build output and example the .pc
   file, you'll see it references /zlib as the location, which is simply where
   I built it in that action. On a developer's machine that's almost certainly
   not going to be where it actually ends up. For example, on the pgAdmin
   build farm, the dependencies all end up in C:\build64\[whatever]. On the
   similar Github action I'm building for PostgreSQL, that artefact will be
   unpacked into /build/zlib.
   
   Of course, for my own builds I can easily make everything use consistent
   directories, however most people who are likely to want to build PostgreSQL
   may not want to also build all the dependencies themselves as well, as some
   are a lot more difficult than zlib. So what tends to happen is people find
   third party builds or upstream official builds.
   
   I would therefore argue that if the .pc file that's found doesn't provide
   correct paths for us, then Meson should fall back to searching in the paths
   specified on its command line for the appropriate libraries/headers (which
   is what it does for OpenSSL for example, as that doesn't include a .pc
   file). This is also what happens with PG16 and earlier.
   
   One other thing I will note is that PG16 and earlier try to use the wrong
   filename for the import library. For years, it's been a requirement to do
   something like this: "copy \zlib\lib\zlib.lib \zlib\lib\zdll.lib" to make a
   build succeed against a "vanilla" zlib build. I haven't got as far as
   figuring out if the same is true with Meson yet.
   
   -- 
   Dave Page
   pgAdmin: https://www.pgadmin.org
   PostgreSQL: https://www.postgresql.org
   EDB: https://www.enterprisedb.com
   

6. Re: zlib detection in Meson on Windows broken?

   Sandeep Thakkar <sandeep.thakkar@enterprisedb.com> — 2024-05-21T14:12:30Z

   Hi Dave,
   
   
   On Tue, May 21, 2024 at 3:12 PM Dave Page <dpage@pgadmin.org> wrote:
   
   > Hi Sandeep, Nazir,
   >
   > On Tue, 21 May 2024 at 10:14, Nazir Bilal Yavuz <byavuz81@gmail.com>
   > wrote:
   >
   >> Hi,
   >>
   >> On Tue, 21 May 2024 at 10:20, Sandeep Thakkar
   >> <sandeep.thakkar@enterprisedb.com> wrote:
   >> >
   >> > Hi Dave,
   >> >
   >> > Is the .pc file generated after the successful build of zlib? If yes,
   >> then meson should be able to detect the installation ideally
   >>
   >> If meson is not able to find the .pc file automatically, using 'meson
   >> setup ... --pkg-config-path $ZLIB_PC_PATH' might help.
   >>
   >
   > The problem is that on Windows there are no standard locations for a
   > Unix-style development library installation such as this, so the chances
   > are that the .pc file will point to entirely the wrong location.
   >
   > For example, please see
   > https://github.com/dpage/winpgbuild/actions/runs/9172187335 which is a
   > Github action that builds a completely vanilla zlib using VC++. If you look
   > at the uploaded artefact containing the build output and example the .pc
   > file, you'll see it references /zlib as the location, which is simply where
   > I built it in that action. On a developer's machine that's almost certainly
   > not going to be where it actually ends up. For example, on the pgAdmin
   > build farm, the dependencies all end up in C:\build64\[whatever]. On the
   > similar Github action I'm building for PostgreSQL, that artefact will be
   > unpacked into /build/zlib.
   >
   >
   The above link returned 404. But I found a successful build at
   https://github.com/dpage/winpgbuild/actions/runs/9175426807. I downloaded
   the artifact but didn't find .pc file as I wanted to look into the content
   of that file.
   
   I had a word with Murali who mentioned he encountered a similar issue while
   building PG17 on windows. He worked-around is by using a template .pc file
   that includes these lines:
   --
   prefix=${pcfiledir}/../..
   exec_prefix=${prefix}
   libdir=${prefix}/lib
   sharedlibdir=${prefix}/lib
   includedir=${prefix}/include
   --
   
   But in general I agree with you on the issue of Meson's dependency on
   pkgconfig files to detect the third party libraries.
   
   Of course, for my own builds I can easily make everything use consistent
   > directories, however most people who are likely to want to build PostgreSQL
   > may not want to also build all the dependencies themselves as well, as some
   > are a lot more difficult than zlib. So what tends to happen is people find
   > third party builds or upstream official builds.
   >
   > I would therefore argue that if the .pc file that's found doesn't provide
   > correct paths for us, then Meson should fall back to searching in the paths
   > specified on its command line for the appropriate libraries/headers (which
   > is what it does for OpenSSL for example, as that doesn't include a .pc
   > file). This is also what happens with PG16 and earlier.
   >
   > One other thing I will note is that PG16 and earlier try to use the wrong
   > filename for the import library. For years, it's been a requirement to do
   > something like this: "copy \zlib\lib\zlib.lib \zlib\lib\zdll.lib" to make a
   > build succeed against a "vanilla" zlib build. I haven't got as far as
   > figuring out if the same is true with Meson yet.
   >
   > --
   > Dave Page
   > pgAdmin: https://www.pgadmin.org
   > PostgreSQL: https://www.postgresql.org
   > EDB: https://www.enterprisedb.com
   >
   >
   
   -- 
   Sandeep Thakkar
   

7. Re: zlib detection in Meson on Windows broken?

   Dave Page <dpage@pgadmin.org> — 2024-05-21T14:24:05Z

   Hi
   
   On Tue, 21 May 2024 at 15:12, Sandeep Thakkar <
   sandeep.thakkar@enterprisedb.com> wrote:
   
   > Hi Dave,
   >
   >
   > On Tue, May 21, 2024 at 3:12 PM Dave Page <dpage@pgadmin.org> wrote:
   >
   >> Hi Sandeep, Nazir,
   >>
   >> On Tue, 21 May 2024 at 10:14, Nazir Bilal Yavuz <byavuz81@gmail.com>
   >> wrote:
   >>
   >>> Hi,
   >>>
   >>> On Tue, 21 May 2024 at 10:20, Sandeep Thakkar
   >>> <sandeep.thakkar@enterprisedb.com> wrote:
   >>> >
   >>> > Hi Dave,
   >>> >
   >>> > Is the .pc file generated after the successful build of zlib? If yes,
   >>> then meson should be able to detect the installation ideally
   >>>
   >>> If meson is not able to find the .pc file automatically, using 'meson
   >>> setup ... --pkg-config-path $ZLIB_PC_PATH' might help.
   >>>
   >>
   >> The problem is that on Windows there are no standard locations for a
   >> Unix-style development library installation such as this, so the chances
   >> are that the .pc file will point to entirely the wrong location.
   >>
   >> For example, please see
   >> https://github.com/dpage/winpgbuild/actions/runs/9172187335 which is a
   >> Github action that builds a completely vanilla zlib using VC++. If you look
   >> at the uploaded artefact containing the build output and example the .pc
   >> file, you'll see it references /zlib as the location, which is simply where
   >> I built it in that action. On a developer's machine that's almost certainly
   >> not going to be where it actually ends up. For example, on the pgAdmin
   >> build farm, the dependencies all end up in C:\build64\[whatever]. On the
   >> similar Github action I'm building for PostgreSQL, that artefact will be
   >> unpacked into /build/zlib.
   >>
   >>
   > The above link returned 404. But I found a successful build at
   > https://github.com/dpage/winpgbuild/actions/runs/9175426807. I downloaded
   > the artifact but didn't find .pc file as I wanted to look into the content
   > of that file.
   >
   
   Yeah, sorry - that was an old one.
   
   Following some offline discussion with Andrew I realised I was building
   zlib incorrectly - using cmake/msbuild instead of nmake and some manual
   copying. Whilst the former does create a working library and a sane looking
   installation, it's not the recommended method, which is documented in a
   very non-obvious way - far less obvious than "oh look, there's a cmake
   config - let's use that".
   
   The new build is done using the recommended method, which works with PG16
   and below out of the box with no need to rename any files.
   
   
   >
   > I had a word with Murali who mentioned he encountered a similar issue
   > while building PG17 on windows. He worked-around is by using a template .pc
   > file that includes these lines:
   > --
   > prefix=${pcfiledir}/../..
   > exec_prefix=${prefix}
   > libdir=${prefix}/lib
   > sharedlibdir=${prefix}/lib
   > includedir=${prefix}/include
   > --
   >
   
   The issue here is that there is no .pc file created with the correct way of
   building zlib, and even if there were (or I created a dummy one),
   pkg-config isn't really a thing on Windows.
   
   I'd also note that from what Andrew has shown me of the zlib installation
   on the buildfarm member drongo, there is no .pc file there either, and yet
   seems to work fine (and my zlib installation now has the exact same set of
   files as his does).
   
   
   >
   > But in general I agree with you on the issue of Meson's dependency on
   > pkgconfig files to detect the third party libraries.
   >
   > Of course, for my own builds I can easily make everything use consistent
   >> directories, however most people who are likely to want to build PostgreSQL
   >> may not want to also build all the dependencies themselves as well, as some
   >> are a lot more difficult than zlib. So what tends to happen is people find
   >> third party builds or upstream official builds.
   >>
   >> I would therefore argue that if the .pc file that's found doesn't provide
   >> correct paths for us, then Meson should fall back to searching in the paths
   >> specified on its command line for the appropriate libraries/headers (which
   >> is what it does for OpenSSL for example, as that doesn't include a .pc
   >> file). This is also what happens with PG16 and earlier.
   >>
   >> One other thing I will note is that PG16 and earlier try to use the wrong
   >> filename for the import library. For years, it's been a requirement to do
   >> something like this: "copy \zlib\lib\zlib.lib \zlib\lib\zdll.lib" to make a
   >> build succeed against a "vanilla" zlib build. I haven't got as far as
   >> figuring out if the same is true with Meson yet.
   >>
   >> --
   >> Dave Page
   >> pgAdmin: https://www.pgadmin.org
   >> PostgreSQL: https://www.postgresql.org
   >> EDB: https://www.enterprisedb.com
   >>
   >>
   >
   > --
   > Sandeep Thakkar
   >
   >
   >
   
   -- 
   Dave Page
   pgAdmin: https://www.pgadmin.org
   PostgreSQL: https://www.postgresql.org
   EDB: https://www.enterprisedb.com
   

8. Re: zlib detection in Meson on Windows broken?

   Andres Freund <andres@anarazel.de> — 2024-05-21T15:04:04Z

   Hi,
   
   On 2024-05-20 11:58:05 +0100, Dave Page wrote:
   > I have very little experience with Meson, and even less interpreting it's
   > logs, but it seems to me that it's not including the extra lib and include
   > directories when it runs the test compile, given the command line it's
   > reporting:
   > 
   > cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c
   > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
   > 
   > Bug, or am I doing something silly?
   
   It's a buglet. We rely on meson's internal fallback detection of zlib, if it's
   not provided via pkg-config or cmake. But it doesn't know about our
   extra_include_dirs parameter. We should probably fix that...
   
   Greetings,
   
   Andres Freund
   
   
   
   

9. Re: zlib detection in Meson on Windows broken?

   Dave Page <dpage@pgadmin.org> — 2024-05-21T15:24:23Z

   On Tue, 21 May 2024 at 16:04, Andres Freund <andres@anarazel.de> wrote:
   
   > Hi,
   >
   > On 2024-05-20 11:58:05 +0100, Dave Page wrote:
   > > I have very little experience with Meson, and even less interpreting it's
   > > logs, but it seems to me that it's not including the extra lib and
   > include
   > > directories when it runs the test compile, given the command line it's
   > > reporting:
   > >
   > > cl
   > C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c
   > > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od
   > /Oi-
   > >
   > > Bug, or am I doing something silly?
   >
   > It's a buglet. We rely on meson's internal fallback detection of zlib, if
   > it's
   > not provided via pkg-config or cmake. But it doesn't know about our
   > extra_include_dirs parameter. We should probably fix that...
   >
   
   Oh good, then I'm not going bonkers. I'm still curious about how it works
   for Andrew but not me, however fixing that buglet should solve my issue,
   and would be sensible behaviour.
   
   Thanks!
   
   -- 
   Dave Page
   pgAdmin: https://www.pgadmin.org
   PostgreSQL: https://www.postgresql.org
   EDB: https://www.enterprisedb.com
   

10. Re: zlib detection in Meson on Windows broken?

    Andres Freund <andres@anarazel.de> — 2024-05-21T17:00:12Z

    Hi,
    
    On 2024-05-20 11:58:05 +0100, Dave Page wrote:
    > I then attempt to build PostgreSQL:
    > 
    >  meson setup build
    > -Dextra_include_dirs=C:/build64/openssl/include,C:/build64/zlib/include
    > -Dextra_lib_dirs=C:/build64/openssl/lib,C:/build64/zlib/lib -Dssl=openssl
    > -Dzlib=enabled --prefix=c:/build64/pgsql
    > 
    > Which results in the output in output.txt, indicating that OpenSSL was
    > correctly found, but zlib was not. I've also attached the meson log.
    
    I forgot to mention that earlier: Assuming you're building something to be
    distributed, I'd recommend --auto-features=enabled/disabled and specifying
    specifically which dependencies you want to be used.
    
    Greetings,
    
    Andres Freund
    
    
    
    

11. Re: zlib detection in Meson on Windows broken?

    Andrew Dunstan <andrew@dunslane.net> — 2024-05-21T19:54:52Z

    On 2024-05-21 Tu 11:04, Andres Freund wrote:
    > Hi,
    >
    > On 2024-05-20 11:58:05 +0100, Dave Page wrote:
    >> I have very little experience with Meson, and even less interpreting it's
    >> logs, but it seems to me that it's not including the extra lib and include
    >> directories when it runs the test compile, given the command line it's
    >> reporting:
    >>
    >> cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c
    >> /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
    >>
    >> Bug, or am I doing something silly?
    > It's a buglet. We rely on meson's internal fallback detection of zlib, if it's
    > not provided via pkg-config or cmake. But it doesn't know about our
    > extra_include_dirs parameter. We should probably fix that...
    >
    
    Yeah. Meanwhile, what I got working on a totally fresh Windows + VS 
    install was instead of using extra_include_dirs etc to add the relevant 
    directories to the environment LIB and INCLUDE settings before calling 
    `"meson setup".
    
    
    cheers
    
    
    andrew
    
    --
    Andrew Dunstan
    EDB: https://www.enterprisedb.com
    
    
    
    
    

12. Re: zlib detection in Meson on Windows broken?

    Tristan Partin <tristan@partin.io> — 2024-05-21T22:09:54Z

    On Tue May 21, 2024 at 10:04 AM CDT, Andres Freund wrote:
    > Hi,
    >
    > On 2024-05-20 11:58:05 +0100, Dave Page wrote:
    > > I have very little experience with Meson, and even less interpreting it's
    > > logs, but it seems to me that it's not including the extra lib and include
    > > directories when it runs the test compile, given the command line it's
    > > reporting:
    > > 
    > > cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c
    > > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
    > > 
    > > Bug, or am I doing something silly?
    >
    > It's a buglet. We rely on meson's internal fallback detection of zlib, if it's
    > not provided via pkg-config or cmake. But it doesn't know about our
    > extra_include_dirs parameter. We should probably fix that...
    
    Here is the relevant Meson code for finding zlib in the Postgres tree:
    
    	postgres_inc_d = ['src/include']
    	postgres_inc_d += get_option('extra_include_dirs')
    	...
    	postgres_inc = [include_directories(postgres_inc_d)]
    	...
    	zlibopt = get_option('zlib')
    	zlib = not_found_dep
    	if not zlibopt.disabled()
    	  zlib_t = dependency('zlib', required: zlibopt)
    
    	  if zlib_t.type_name() == 'internal'
    		# if fallback was used, we don't need to test if headers are present (they
    		# aren't built yet, so we can't test)
    		zlib = zlib_t
    	  elif not zlib_t.found()
    		warning('did not find zlib')
    	  elif not cc.has_header('zlib.h',
    		  args: test_c_args, include_directories: postgres_inc,
    		  dependencies: [zlib_t], required: zlibopt)
    		warning('zlib header not found')
    	  elif not cc.has_type('z_streamp',
    		  dependencies: [zlib_t], prefix: '#include <zlib.h>',
    		  args: test_c_args, include_directories: postgres_inc)
    		if zlibopt.enabled()
    		  error('zlib version is too old')
    		else
    		  warning('zlib version is too old')
    		endif
    	  else
    		zlib = zlib_t
    	  endif
    
    	  if zlib.found()
    		cdata.set('HAVE_LIBZ', 1)
    	  endif
    	endif
    
    You can see that we do pass the include dirs to the has_header check. 
    Something seems to be going wrong here since your extra_include_dirs 
    isn't being properly translated to include arguments.
    
    -- 
    Tristan Partin
    https://tristan.partin.io
    
    
    
    

13. Re: zlib detection in Meson on Windows broken?

    Dave Page <dpage@pgadmin.org> — 2024-05-22T11:12:01Z

    On Tue, 21 May 2024 at 18:00, Andres Freund <andres@anarazel.de> wrote:
    
    > Hi,
    >
    > On 2024-05-20 11:58:05 +0100, Dave Page wrote:
    > > I then attempt to build PostgreSQL:
    > >
    > >  meson setup build
    > > -Dextra_include_dirs=C:/build64/openssl/include,C:/build64/zlib/include
    > > -Dextra_lib_dirs=C:/build64/openssl/lib,C:/build64/zlib/lib -Dssl=openssl
    > > -Dzlib=enabled --prefix=c:/build64/pgsql
    > >
    > > Which results in the output in output.txt, indicating that OpenSSL was
    > > correctly found, but zlib was not. I've also attached the meson log.
    >
    > I forgot to mention that earlier: Assuming you're building something to be
    > distributed, I'd recommend --auto-features=enabled/disabled and specifying
    > specifically which dependencies you want to be used.
    >
    
    Good idea - thanks.
    
    -- 
    Dave Page
    pgAdmin: https://www.pgadmin.org
    PostgreSQL: https://www.postgresql.org
    EDB: https://www.enterprisedb.com
    

14. Re: zlib detection in Meson on Windows broken?

    Dave Page <dpage@pgadmin.org> — 2024-05-22T11:12:48Z

    On Tue, 21 May 2024 at 20:54, Andrew Dunstan <andrew@dunslane.net> wrote:
    
    >
    > On 2024-05-21 Tu 11:04, Andres Freund wrote:
    > > Hi,
    > >
    > > On 2024-05-20 11:58:05 +0100, Dave Page wrote:
    > >> I have very little experience with Meson, and even less interpreting
    > it's
    > >> logs, but it seems to me that it's not including the extra lib and
    > include
    > >> directories when it runs the test compile, given the command line it's
    > >> reporting:
    > >>
    > >> cl
    > C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c
    > >> /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od
    > /Oi-
    > >>
    > >> Bug, or am I doing something silly?
    > > It's a buglet. We rely on meson's internal fallback detection of zlib,
    > if it's
    > > not provided via pkg-config or cmake. But it doesn't know about our
    > > extra_include_dirs parameter. We should probably fix that...
    > >
    >
    > Yeah. Meanwhile, what I got working on a totally fresh Windows + VS
    > install was instead of using extra_include_dirs etc to add the relevant
    > directories to the environment LIB and INCLUDE settings before calling
    > `"meson setup".
    >
    
    Yes, that works for me too.
    
    -- 
    Dave Page
    pgAdmin: https://www.pgadmin.org
    PostgreSQL: https://www.postgresql.org
    EDB: https://www.enterprisedb.com
    

15. Re: zlib detection in Meson on Windows broken?

    Nazir Bilal Yavuz <byavuz81@gmail.com> — 2024-05-22T13:11:09Z

    Hi,
    
    On Tue, 21 May 2024 at 18:24, Dave Page <dpage@pgadmin.org> wrote:
    >
    >
    >
    > On Tue, 21 May 2024 at 16:04, Andres Freund <andres@anarazel.de> wrote:
    >>
    >> Hi,
    >>
    >> On 2024-05-20 11:58:05 +0100, Dave Page wrote:
    >> > I have very little experience with Meson, and even less interpreting it's
    >> > logs, but it seems to me that it's not including the extra lib and include
    >> > directories when it runs the test compile, given the command line it's
    >> > reporting:
    >> >
    >> > cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c
    >> > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
    >> >
    >> > Bug, or am I doing something silly?
    >>
    >> It's a buglet. We rely on meson's internal fallback detection of zlib, if it's
    >> not provided via pkg-config or cmake. But it doesn't know about our
    >> extra_include_dirs parameter. We should probably fix that...
    >
    >
    > Oh good, then I'm not going bonkers. I'm still curious about how it works for Andrew but not me, however fixing that buglet should solve my issue, and would be sensible behaviour.
    >
    > Thanks!
    
    I tried to install your latest zlib artifact (nmake one) to the
    Windows CI images (not the official ones) [1]. Then, I used the
    default meson.build file to build but meson could not find the zlib.
    After that, I modified it like you suggested before; I used a
    'cc.find_library()' to find zlib as a fallback method and it seems it
    worked [2]. Please see meson setup logs below [3], does something
    similar to the attached solve your problem?
    
    The interesting thing is, I also tried this 'cc.find_library' method
    with your old artifact (cmake one). It was able to find zlib but all
    tests failed [4].
    
    Experimental zlib meson.build diff is attached.
    
    [1] https://cirrus-ci.com/task/6736867247259648
    [2] https://cirrus-ci.com/build/5286228755480576
    [3]
    Run-time dependency zlib found: NO (tried pkgconfig, cmake and system)
    Has header "zlib.h" : YES
    Library zlib found: YES
    ...
      External libraries
    ...
        zlib                   : YES
    ...
    [4] https://cirrus-ci.com/task/5208433811521536
    
    -- 
    Regards,
    Nazir Bilal Yavuz
    Microsoft
    

16. Re: zlib detection in Meson on Windows broken?

    Dave Page <dpage@pgadmin.org> — 2024-05-22T14:20:56Z

    Hi
    
    On Wed, 22 May 2024 at 14:11, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
    
    > Hi,
    >
    > On Tue, 21 May 2024 at 18:24, Dave Page <dpage@pgadmin.org> wrote:
    > >
    > >
    > >
    > > On Tue, 21 May 2024 at 16:04, Andres Freund <andres@anarazel.de> wrote:
    > >>
    > >> Hi,
    > >>
    > >> On 2024-05-20 11:58:05 +0100, Dave Page wrote:
    > >> > I have very little experience with Meson, and even less interpreting
    > it's
    > >> > logs, but it seems to me that it's not including the extra lib and
    > include
    > >> > directories when it runs the test compile, given the command line it's
    > >> > reporting:
    > >> >
    > >> > cl
    > C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c
    > >> > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od
    > /Oi-
    > >> >
    > >> > Bug, or am I doing something silly?
    > >>
    > >> It's a buglet. We rely on meson's internal fallback detection of zlib,
    > if it's
    > >> not provided via pkg-config or cmake. But it doesn't know about our
    > >> extra_include_dirs parameter. We should probably fix that...
    > >
    > >
    > > Oh good, then I'm not going bonkers. I'm still curious about how it
    > works for Andrew but not me, however fixing that buglet should solve my
    > issue, and would be sensible behaviour.
    > >
    > > Thanks!
    >
    > I tried to install your latest zlib artifact (nmake one) to the
    > Windows CI images (not the official ones) [1]. Then, I used the
    > default meson.build file to build but meson could not find the zlib.
    > After that, I modified it like you suggested before; I used a
    > 'cc.find_library()' to find zlib as a fallback method and it seems it
    > worked [2]. Please see meson setup logs below [3], does something
    > similar to the attached solve your problem?
    >
    
    That patch does solve my problem - thank you!
    
    
    >
    > The interesting thing is, I also tried this 'cc.find_library' method
    > with your old artifact (cmake one). It was able to find zlib but all
    > tests failed [4].
    >
    
    Very odd. Whilst I haven't used that particular build elsewhere, we've been
    building PostgreSQL and shipping client utilities with pgAdmin using
    cmake-built zlib for years.
    
    
    >
    > Experimental zlib meson.build diff is attached.
    >
    > [1] https://cirrus-ci.com/task/6736867247259648
    > [2] https://cirrus-ci.com/build/5286228755480576
    > [3]
    > Run-time dependency zlib found: NO (tried pkgconfig, cmake and system)
    > Has header "zlib.h" : YES
    > Library zlib found: YES
    > ...
    >   External libraries
    > ...
    >     zlib                   : YES
    > ...
    > [4] https://cirrus-ci.com/task/5208433811521536
    >
    > --
    > Regards,
    > Nazir Bilal Yavuz
    > Microsoft
    >
    
    
    -- 
    Dave Page
    pgAdmin: https://www.pgadmin.org
    PostgreSQL: https://www.postgresql.org
    EDB: https://www.enterprisedb.com
    

17. Re: zlib detection in Meson on Windows broken?

    Nazir Bilal Yavuz <byavuz81@gmail.com> — 2024-05-22T16:49:50Z

    Hi,
    
    On Wed, 22 May 2024 at 17:21, Dave Page <dpage@pgadmin.org> wrote:
    >
    > Hi
    >
    > On Wed, 22 May 2024 at 14:11, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
    >>
    >>
    >> I tried to install your latest zlib artifact (nmake one) to the
    >> Windows CI images (not the official ones) [1]. Then, I used the
    >> default meson.build file to build but meson could not find the zlib.
    >> After that, I modified it like you suggested before; I used a
    >> 'cc.find_library()' to find zlib as a fallback method and it seems it
    >> worked [2]. Please see meson setup logs below [3], does something
    >> similar to the attached solve your problem?
    >
    >
    > That patch does solve my problem - thank you!
    
    I am glad that it worked!
    
    Do you think that we need to have this patch in the upstream Postgres?
    I am not sure because:
    - There is a case that meson is able to find zlib but tests fail.
    - This might be a band-aid fix rather than a permanent fix.
    
    -- 
    Regards,
    Nazir Bilal Yavuz
    Microsoft
    
    
    
    

18. Re: zlib detection in Meson on Windows broken?

    Dave Page <dpage@pgadmin.org> — 2024-05-22T17:18:07Z

    On Wed, 22 May 2024 at 17:50, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
    
    > Hi,
    >
    > On Wed, 22 May 2024 at 17:21, Dave Page <dpage@pgadmin.org> wrote:
    > >
    > > Hi
    > >
    > > On Wed, 22 May 2024 at 14:11, Nazir Bilal Yavuz <byavuz81@gmail.com>
    > wrote:
    > >>
    > >>
    > >> I tried to install your latest zlib artifact (nmake one) to the
    > >> Windows CI images (not the official ones) [1]. Then, I used the
    > >> default meson.build file to build but meson could not find the zlib.
    > >> After that, I modified it like you suggested before; I used a
    > >> 'cc.find_library()' to find zlib as a fallback method and it seems it
    > >> worked [2]. Please see meson setup logs below [3], does something
    > >> similar to the attached solve your problem?
    > >
    > >
    > > That patch does solve my problem - thank you!
    >
    > I am glad that it worked!
    >
    > Do you think that we need to have this patch in the upstream Postgres?
    > I am not sure because:
    > - There is a case that meson is able to find zlib but tests fail.
    > - This might be a band-aid fix rather than a permanent fix.
    
    
    Yes I do:
    
    - This is the documented way to build/install zlib on Windows.
    - The behaviour with the patch matches <= PG16
    - The behaviour with the patch is consistent with OpenSSL detection, and
    (from a quick, unrelated test), libxml2 detection.
    
    Thanks!
    
    >