Horton

Re: Windows pg_basebackup unable to create >2GB pg_wal.tar tarballs ("could not close file: Invalid argument" when creating pg_wal.tar of size ~ 2^31 bytes)

Thomas Munro <thomas.munro@gmail.com>

View thread View on postgr.es
From: Thomas Munro <thomas.munro@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Jakub Wartak <jakub.wartak@enterprisedb.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Davinder Singh <davinder.singh@enterprisedb.com>, Dilip Kumar <dilip.kumar@enterprisedb.com>
Date: 2024-12-06T06:01:06Z
Lists: pgsql-hackers

Attachments

Some better new:

1.  _chsize_s does in fact seem to work in an msvcrt.dll build.
Thanks to Andres for testing that for me on Windows with a standalone
2 line program on ucrt and msvcrt runtimes.

2.  MinGW always has _chsize_s, but it does more or less what I had
been proposing already if it can't find the function by the Windows
thing like dlsym, so there's no point in writing another thing like that.
The test was with a 3G file though so it was the real function.

https://github.com/mingw-w64/mingw-w64/blob/master/mingw-w64-crt/secapi/_chsize_s.c

3.  After a long time trying various ways to make it tidy and getting
incomprehensible results I eventually realised that precompiled
headers were causing strange results on MinGW/Meson builds (on CI),
due to fighting over macros with this guy:

https://github.com/mingw-w64/mingw-w64/blob/master/mingw-w64-headers/crt/unistd.h

I found it easiest to completely stop it from getting in our way
completely, with the attached.  Thoughts, anyone?

Davinder, Jakub, are you in a position repro the issue and make
versions of the patch for the 13, 15 and master branches on top of
that?

Commits

  1. Fix off_t overflow in pg_basebackup on Windows.

  2. Provide 64-bit ftruncate() and lseek() on Windows.

  3. Require ucrt if using MinGW.