Horton

Re: optimizing pg_upgrade's once-in-each-database steps

Corey Huinker <corey.huinker@gmail.com>

View thread View on postgr.es
From: Corey Huinker <corey.huinker@gmail.com>
To: Nathan Bossart <nathandbossart@gmail.com>
Cc: Ilya Gladyshev <ilya.v.gladyshev@gmail.com>, Daniel Gustafsson <daniel@yesql.se>, Robert Haas <robertmhaas@gmail.com>, Jeff Davis <pgsql@j-davis.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2024-08-08T22:18:38Z
Lists: pgsql-hackers 
On Tue, Aug 6, 2024 at 3:20 PM Nathan Bossart <nathandbossart@gmail.com>
wrote:

> On Sun, Aug 04, 2024 at 07:19:57PM +0100, Ilya Gladyshev wrote:
> > -- End of review --
>
> Thanks for the review.  I've attempted to address all your feedback in v8
> of the patch set.  I think the names could still use some work, but I
> wanted to get the main structure in place before trying to fix them.
>

I think the underlying mechanism is basically solid, but I have one
question: isn't this the ideal case for using libpq pipelining? That would
allow subsequent tasks to launch while the main loop slowly gets around to
clearing off completed tasks on some other connection.

Commits

  1. Introduce framework for parallelizing various pg_upgrade tasks.

  2. pg_upgrade: Parallelize WITH OIDS check.

  3. pg_upgrade: Parallelize contrib/isn check.

  4. pg_upgrade: Parallelize data type checks.

  5. pg_upgrade: Parallelize encoding conversion check.

  6. pg_upgrade: Parallelize incompatible polymorphics check.

  7. pg_upgrade: Parallelize postfix operator check.

  8. pg_upgrade: Parallelize retrieving extension updates.

  9. pg_upgrade: Parallelize retrieving loadable libraries.

  10. pg_upgrade: Parallelize retrieving relation information.

  11. pg_upgrade: Parallelize subscription check.

  12. pg_upgrade: Move live_check variable to user_opts.

  13. pg_upgrade: run all data type checks per connection