Re: patch for parallel pg_dump

On 03/28/2012 03:17 PM, Andrew Dunstan wrote:
>
>
> On 03/28/2012 02:27 PM, Robert Haas wrote:
>> On Wed, Mar 28, 2012 at 2:20 PM, Alvaro Herrera
>> <alvherre@commandprompt.com>  wrote:
>>> Excerpts from Robert Haas's message of mié mar 28 14:46:30 -0300 2012:
>>>> I keep hoping someone who knows Windows is going to take a look at
>>>> this, but so far no luck.  It could also really use some attention
>>>> from someone who has an actual really big database handy, to see how
>>>> successful it is in reducing the dump time.  Without those things, I
>>>> can't see this getting committed.  But in the meantime, a few fairly
>>>> minor comments based on reading the code.
>>> My main comment about the current patch is that it looks like it's
>>> touching pg_restore parallel code by moving some stuff into parallel.c.
>>> If that's really the case and its voluminous, maybe this patch would
>>> shrink a bit if we could do the code moving in a first patch.  That
>>> would be mostly mechanical.  Then the interesting stuff would apply on
>>> top of that.  That would make review easier.
>> +1.
>
> +1 also.
>
> FYI I am just starting some test runs on Windows. I also have a 
> reasonably sized db (98 gb) on my SL6 server which should be perfect 
> for testing this (I just partitioned its two main tables), and will 
> try to get some timing runs.


First hurdle: It doesn't build under Windows/mingw-w64:

    x86_64-w64-mingw32-gcc -O2 -Wall -Wmissing-prototypes
    -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels
    -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing
    -fwrapv -fexcess-precision=standard -g
    -I../../../src/interfaces/libpq -I../../../src/include
    -I./src/include/port/win32 -DEXEC_BACKEND 
    -I/c/prog/mingwdep/include "-I../../../src/include/port/win32"  -c
    -o parallel.o parallel.c
    parallel.c:40:12: error: static declaration of 'pgpipe' follows
    non-static declaration
    ../../../src/include/port.h:268:12: note: previous declaration of
    'pgpipe' was here
    parallel.c:41:12: error: static declaration of 'piperead' follows
    non-static declaration
    ../../../src/include/port.h:269:12: note: previous declaration of
    'piperead' was here
    parallel.c: In function 'ParallelBackupStart':
    parallel.c:455:9: warning: passing argument 3 of '_beginthreadex'
    from incompatible pointer type
    c:\mingw64\bin\../lib/gcc/x86_64-w64-mingw32/4.5.3/../../../../x86_64-w64-mingw32/include/process.h:31:29:
    note: expected 'unsigned int (*)(void *)' but argument is of type
    'unsigned int (*)(struct WorkerInfo *)'
    make[3]: *** [parallel.o] Error 1
    make[3]: Leaving directory
    `/home/andrew/bf/root/HEAD/pgsql/src/bin/pg_dump'
    make[2]: *** [all-pg_dump-recurse] Error 2
    make[2]: Leaving directory `/home/andrew/bf/root/HEAD/pgsql/src/bin'
    make[1]: *** [all-bin-recurse] Error 2
    make[1]: Leaving directory `/home/andrew/bf/root/HEAD/pgsql/src'
    make: *** [all-src-recurse] Error 2

I'll have a look at that in a little while.

cheers

andrew


>
> cheers
>
> andrew
>
>