pg_upgrade with parallel tablespace copying

Bruce Momjian <bruce@momjian.us>

From: Bruce Momjian <bruce@momjian.us>
To: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Date: 2013-01-08T03:51:21Z
Lists: pgsql-hackers

Attachments

Pg_upgrade by default (without --link) copies heap/index files from the
old to new cluster.  This patch implements parallel heap/index file
copying in pg_upgrade using the --jobs option.  It uses the same
infrastructure used for pg_upgrade parallel dump/restore.  Here are the
performance results:

	           --- seconds ---
	   GB        git    patched
	    2       62.09    63.75
	    4       95.93   107.22
	    8      194.96   195.29
	   16      494.38   348.93
	   32      983.28   644.23
	   64     2227.73  1244.08
	  128     4735.83  2547.09

Because of the kernel cache, you only see a big win when the amount of
copy data exceeds the kernel cache.  For testing, I used a 24GB, 16-core
machine with two magnetic disks with one tablespace on each.  Using more
tablespaces would yield larger improvements.  My test script is
attached.  

I consider this patch ready for application.  This is the last
pg_upgrade performance improvement idea I am considering.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + It's impossible for everything to be true. +