Horton

Re: Windows pg_basebackup unable to create >2GB pg_wal.tar tarballs ("could not close file: Invalid argument" when creating pg_wal.tar of size ~ 2^31 bytes)

Thomas Munro <thomas.munro@gmail.com>

View thread View on postgr.es
From: Thomas Munro <thomas.munro@gmail.com>
To: Jakub Wartak <jakub.wartak@enterprisedb.com>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Davinder Singh <davinder.singh@enterprisedb.com>, Dilip Kumar <dilip.kumar@enterprisedb.com>
Date: 2024-11-21T13:38:09Z
Lists: pgsql-hackers 
On Thu, Nov 21, 2024 at 11:44 PM Jakub Wartak
<jakub.wartak@enterprisedb.com> wrote:
> This literally looks like something like off_t/size_t would be limited to 2^31 somewhere.

off_t is 32 bits on Windows.  I'd be quite suspicious of the
arithmetic involving 'currpos'.  What happens if you change all the
off_t in walmethods.c/.h to pgoff_t?  The lseek() is suspicious too,
and might need to be redirected to _lseeki64().

Commits

  1. Fix off_t overflow in pg_basebackup on Windows.

  2. Provide 64-bit ftruncate() and lseek() on Windows.

  3. Require ucrt if using MinGW.