Re: pg_dump and thousands of schemas

Jeff Janes <jeff.janes@gmail.com>

From: Jeff Janes <jeff.janes@gmail.com>

To: Tatsuo Ishii <ishii@postgresql.org>

Cc: tgl@sss.pgh.pa.us, pgsql-performance@postgresql.org

Date: 2012-06-10T23:47:41Z

Lists: pgsql-hackers, pgsql-performance

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Teach AbortOutOfAnyTransaction to clean up partially-started transactions.
- 2755abf386e6 9.2.0 cited

On Wed, May 30, 2012 at 2:06 AM, Tatsuo Ishii <ishii@postgresql.org> wrote:
>> Yeah, Jeff's experiments indicated that the remaining bottleneck is lock
>> management in the server.  What I fixed so far on the pg_dump side
>> should be enough to let partial dumps run at reasonable speed even if
>> the whole database contains many tables.  But if psql is taking
>> AccessShareLock on lots of tables, there's still a problem.
>
> Ok, I modified the part of pg_dump where tremendous number of LOCK
> TABLE are issued. I replace them with single LOCK TABLE with multiple
> tables. With 100k tables LOCK statements took 13 minutes in total, now
> it only takes 3 seconds. Comments?

Could you rebase this?  I tried doing it myself, but must have messed
it up because it got slower rather than faster.

Thanks,

Jeff