Horton

Re: pg_dump --split patch

Joel Jacobson <joel@gluefinance.com>

View thread
From: Joel Jacobson <joel@gluefinance.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Aidan Van Dyk <aidan@highrise.ca>, Gurjeet Singh <singh.gurjeet@gmail.com>, pgsql-hackers@postgresql.org, Andrew Dunstan <andrew@dunslane.net>, David Wilson <david.t.wilson@gmail.com>
Date: 2010-12-29T15:21:16Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. pg_dump: Output functions deterministically sorted

  2. When sorting functions in pg_dump, break ties (same name) by number of arguments

2010/12/29 Tom Lane <tgl@sss.pgh.pa.us>

> I think they're fundamentally different things, because the previously
> proposed patch is an extension of the machine-readable archive format,
> and has to remain so because of the expectation that people will want
> to use parallel restore with it.  Joel is arguing for a split-up of
> the text dump format.
>
>
Yes, exactly.

My patch is of course also a lot smaller :-)
pg_dump-directory.diff.........................: 112 853 bytes
pg-dump-split-plain-text-files-9.1devel.patch..:   5 579 bytes

I just tried the pg_dump-directory.diff patch.
The only thing is has in common with my patch is it writes data to different
files, and it's only the data which is splitted into different files, the
schema appears to go into the single file "TOC".

Example, pg_dump-directory.diff:

$ ./pg_dump -f /crypt/dirpatch -F d -s glue
$ ls -la /crypt/dirpatch/
TOC
(1 file)

$ rm -rf /crypt/dirpatch

$ ./pg_dump -f /crypt/dirpatch -F d glue

$ ls /crypt/dirpatch/
6503.dat
6504.dat
...lots of files...
6871.dat
6872.dat
6873.dat
6874.dat
TOC

Example, pg_dump --split patch:

$ pg_dump -f /crypt/splitpatch -F p --split -s glue

$ ls /crypt/splitpatch*
/crypt/splitpatch (file)
/crypt/splitpatch-split: (directory)
    myschema1
    myschema2
    public
$ ls /crypt/splitpatch-split/public/
AGGREGATE
CONSTRAINT
FK_CONSTRAINT
FUNCTION
INDEX
SEQUENCE
TABLE
TRIGGER
TYPE
VIEW

$ ls /crypt/splitpatch-split/public/FUNCTION/
    myfunc.sql
    otherfunc.sql

$ cat /crypt/splitpatch
--
-- PostgreSQL database dump
--

SET statement_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = off;
SET check_function_bodies = false;
SET client_min_messages = warning;
SET escape_string_warning = off;
...etc...
\i /crypt/splitpatch-split/public/FUNCTION/myfunc.sql
\i /crypt/splitpatch-split/public/FUNCTION/otherfunc.sql


-- 
Best regards,

Joel Jacobson
Glue Finance