Re: parallel pg_restore - WIP patch

Stefan Kaltenbrunner wrote:
> Andrew Dunstan wrote:
>>
>> Attached is my WIP patch for parallel pg_restore. It's still very 
>> rough, but seems to work.
>>
>> Anyone who can test this with highend equipment would be helping some.
>
> tried playing with this(on a 22Gb compressed dump using 4 connections) 
> but it does not seem to work at all for me:
>
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] could not uncompress data: invalid block 
> type
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] could not uncompress data: invalid 
> stored block lengths
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] could not uncompress data: invalid 
> distance too far back
> pg_restore: [custom archiver] could not uncompress data: invalid 
> distances set
> pg_restore: [custom archiver] could not uncompress data: invalid code 
> lengths set
> pg_restore: [custom archiver] could not uncompress data: incorrect 
> data check
> pg_restore: [custom archiver] could not uncompress data: invalid code 
> lengths set
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] out of memory
> pg_restore: [custom archiver] could not uncompress data: invalid 
> literal/length code
> pg_restore: [custom archiver] could not uncompress data: invalid 
> literal/length code
> pg_restore: [custom archiver] could not uncompress data: invalid block 
> type
>
> each pg_restore process seem to eat a few gigabytes of memory in a few 
> seconds.


Ouch. Ok, Thanks for the report. I will investigate.

cheers

andrew