Horton

Re: optimizing pg_upgrade's once-in-each-database steps

Nathan Bossart <nathandbossart@gmail.com>

View thread View on postgr.es
From: Nathan Bossart <nathandbossart@gmail.com>
To: Ilya Gladyshev <ilya.v.gladyshev@gmail.com>
Cc: Corey Huinker <corey.huinker@gmail.com>, Daniel Gustafsson <daniel@yesql.se>, Robert Haas <robertmhaas@gmail.com>, Jeff Davis <pgsql@j-davis.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2024-09-04T01:41:49Z
Lists: pgsql-hackers

Attachments

On Wed, Sep 04, 2024 at 12:28:23AM +0100, Ilya Gladyshev wrote:
> The fix looks right to me, but I got confused by the skip_wait and this
> `if`:
> 
> +            if (PQstatus(slot->conn) != CONNECTION_OK)
> +                return;
> 
> This branch checks connection status that hasn't been refreshed after the
> select. When we go back to wait_slots after this, PQconnectPoll will refresh
> the connection status and run select with skip_wait=true, I believe, we
> could simplify this by moving the PQconnectPoll back to the process_slots,
> so that we can process connection right after polling, if it's ready.

Ah, yes, that's a nice way to simplify things.  I ended up just making it
process_slot()'s responsibility to set the correct select_mode, at which
point the logic in the switch statement in wait_on_slots() is sparse enough
that it seems better to convert it to a couple of short "if" statements.

-- 
nathan

Commits

  1. Introduce framework for parallelizing various pg_upgrade tasks.

  2. pg_upgrade: Parallelize WITH OIDS check.

  3. pg_upgrade: Parallelize contrib/isn check.

  4. pg_upgrade: Parallelize data type checks.

  5. pg_upgrade: Parallelize encoding conversion check.

  6. pg_upgrade: Parallelize incompatible polymorphics check.

  7. pg_upgrade: Parallelize postfix operator check.

  8. pg_upgrade: Parallelize retrieving extension updates.

  9. pg_upgrade: Parallelize retrieving loadable libraries.

  10. pg_upgrade: Parallelize retrieving relation information.

  11. pg_upgrade: Parallelize subscription check.

  12. pg_upgrade: Move live_check variable to user_opts.

  13. pg_upgrade: run all data type checks per connection