Re: pg_upgrade test failure

Thomas Munro <thomas.munro@gmail.com>

From: Thomas Munro <thomas.munro@gmail.com>
To: Justin Pryzby <pryzby@telsasoft.com>
Cc: Andres Freund <andres@anarazel.de>, Michael Paquier <michael@paquier.xyz>, pgsql-hackers@lists.postgresql.org, Nazir Bilal Yavuz <byavuz81@gmail.com>
Date: 2023-01-31T20:54:42Z
Lists: pgsql-hackers

Attachments

On Wed, Feb 1, 2023 at 6:28 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > I pushed the rmtree() change.  Let's see if that helps, or tells us
> > something new.
>
> I found a few failures since then:
>
> https://api.cirrus-ci.com/v1/artifact/task/6696942420361216/testrun/build/testrun/pg_upgrade/002_pg_upgrade/log/regress_log_002_pg_upgrade
>
> pg_upgrade: warning: could not remove directory "C:/cirrus/build/testrun/pg_upgrade/002_pg_upgrade/data/t_002_pg_upgrade_new_node_data/pgdata/pg_upgrade_output.d/20230131T134931.720/log": Directory not empty
> pg_upgrade: warning: could not remove directory "C:/cirrus/build/testrun/pg_upgrade/002_pg_upgrade/data/t_002_pg_upgrade_new_node_data/pgdata/pg_upgrade_output.d/20230131T134931.720": Directory not empty

So no change: we didn't see "could not unlink file ...".  So I think
that means that it was rmtree() that unlinked the file for the *first*
time, but someone else has it open.

Even though Windows is at this point eroding my love of computers and
making me consider a new career in, I dunno, carrot farming or
something, I have one more idea.  Check out this kluge in
src/bin/pg_upgrade/exec.c:

        /*
         * "pg_ctl -w stop" might have reported that the server has stopped
         * because the postmaster.pid file has been removed, but "pg_ctl -w
         * start" might still be in the process of closing and might still be
         * holding its stdout and -l log file descriptors open.  Therefore,
         * try to open the log file a few more times.
         */

I'm not sure about anything, but if that's what's happening here, then
maybe the attached would help.  In short, it would make the previous
theory true (the idea of a second unlink() saving the day).

Commits

  1. ci: freebsd: stop postgres instance on failure, to prevent log upload failures

  2. Try to fix pg_upgrade test on Windows, again.

  3. Refactor rmtree() to use get_dirent_type().

  4. Fix unlink() for STATUS_DELETE_PENDING on Windows.

  5. Move any remaining files generated by pg_upgrade into an internal subdir

  6. Make unlink() work for junction points on Windows.

  7. pg_upgrade: Move all the files generated internally to a subdirectory

  8. In pg_upgrade, try a few times to open a log file.