Re: Further pg_upgrade analysis for many tables

On 11/14/2012 10:08 AM, Bruce Momjian wrote:
> On Wed, Nov 14, 2012 at 06:11:27AM +0200, Ants Aasma wrote:
>>
>> I agree that parallel restore for schemas is a hard problem. But I
>> didn't mean parallelism within the restore, I meant that we could
>> start both postmasters and pipe the output from dump directly to
>> restore. This way the times for dumping and restoring can overlap.
> Wow, that is a very creative idea.  The current code doesn't do that,
> but this has the potential of doubling pg_upgrade's speed, without
> adding a lot of complexity.  Here are the challenges of this approach:
>
> *  I would need to log the output of pg_dumpall as it is passed to psql
> so users can debug problems


Instead of piping it directly, have pg_upgrade work as a tee, pumping 
bytes both to psql and a file. This doesn't seem terribly hard.

>
> *  pg_upgrade never runs the old and new clusters at the same time for
> fear that it will run out of resources, e.g. shared memory, or if they
> are using the same port number.  We can make this optional and force
> different port numbers.


Right.

cheers

andrew