Re: parallel pg_restore - WIP patch

Andrew Dunstan <andrew@dunslane.net>

From: Andrew Dunstan <andrew@dunslane.net>
To: Philip Warner <pjw@rhyme.com.au>
Cc: Stefan Kaltenbrunner <stefan@kaltenbrunner.cc>, Tom Lane <tgl@sss.pgh.pa.us>, Russell Smith <mr-russ@pws.com.au>, PostgreSQL-development <pgsql-hackers@postgresql.org>, Jeffrey Baker <jwbaker@gmail.com>
Date: 2008-09-30T12:55:47Z
Lists: pgsql-hackers

Philip Warner wrote:
> Andrew Dunstan wrote:
>   
>> Unfortunately, it quite possibly would. You would not be able to build
>> two indexes on the same table in parallel, even though they wouldn't
>> have conflicting locks.
>>     
> I suppose so, but:
>
> 1. By the same logic it might speed things up; it might build two
> completely separate indexes and thereby avoid (some kind of) contention.
> In any case, it would most likely do *something* else. It should only
> reduce performance if (a) it can do nothing or (b) there is a benefit in
> building multiple indexes on the same table at the same time.
>
> 2. Perhaps if there are a limited number of items that share
> dependencies but which are known to be OK (ie. indexes), maybe list them
> in the inner loop as exceptions and allow them to run parallel. This
> would mean a failure to list a new TOC item type would result in worse
> performance rather than a crash.
>
>
>   

I will look at it in due course. Right now my concern is simply to get 
something that works that we can do some testing with. I think that's 
what we have now (fingers crossed). Some parts of it are jury rigged.

BTW, though, building indexes for the same table together is likely to 
be a win AIUI, especially given the recent work on synchronised scans.

cheers

andrew