Re: Windows pg_basebackup unable to create >2GB pg_wal.tar tarballs ("could not close file: Invalid argument" when creating pg_wal.tar of size ~ 2^31 bytes)

Hi,

On Sat, Nov 23, 2024 at 5:14 AM Thomas Munro <thomas.munro@gmail.com> wrote:

> On Fri, Nov 22, 2024 at 10:55 PM Jakub Wartak
> <jakub.wartak@enterprisedb.com> wrote:
> > but with attached _lseeki64 dirty patch I've got success and resulting
> tarball greater than 2^31 too:
>


>
> For ftruncate(), though you didn't see a problem with that, Jacob is
> quite right that it must be able to corrupt stuff, and I retract that
> replacement code I showed earlier, it's not really OK to move the file
> position around.  I would like to use _s_chsize() instead, but it
> arrived in msvcr80.dll so I don't think old MinGW can use it.  Here's
> a new idea (sketch code not tested):
>
> static inline int
> ftruncate(int fd, pgoff_t length)
> {
> #if defined(_UCRT) || defined(_MSC_VER)
>     /* MinGW + UCRT and all supported MSVC versions have this. */
>     errno = _s_chsize(fd, length);
>     return errno == 0 ? 0 : -1;
> #else
>     /* MinGW + ancient msvcrt.dll has only _chsize, limited by off_t
> (long). */
>     if (length > LONG_MAX)
>     {
>         /* A clear error is better than silent corruption. */
>         errno = EFBIG;
>         return - 1;
>     }
>     return _chsize(fd, length);
> #endif
> }
>

With the earlier patch, there were some compilation issues on other
platforms, I have modified the patch a little by using some of the changes
from the earlier patch shared by Thomas. Also, I have used the above code
to enable ftruncate() to handle pgoff_t. All the builds and tests on the CI
build <https://cirrus-ci.com/build/6351585205288960> passed except an
already existing build failure in the mingw_cross compiler (not related to
the patch).

-- 
Regards,
Davinder.