Horton

Re: 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

Sergey Koposov <koposov@ast.cam.ac.uk>

View thread
From: Sergey Koposov <koposov@ast.cam.ac.uk>
To: Jeff Janes <jeff.janes@gmail.com>
Cc: Florian Pflug <fgp@phlo.org>, Merlin Moncure <mmoncure@gmail.com>, Robert Haas <robertmhaas@gmail.com>, pgsql-hackers@postgresql.org, Stephen Frost <sfrost@snowman.net>
Date: 2012-06-01T11:39:14Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Improve bulk-insert performance by keeping the current target buffer pinned

On Thu, 31 May 2012, Jeff Janes wrote:

>>
>> No, idt_match is getting filled by multi-threaded copy() and then joined
>> with 4 other big tables like idt_phot. The result is then split into
>> partitions.
>
> That does make things more complicated.  But you could you partition
> it at that level and then do the joins partition-wise?

Unfortunately the information to understand in which partition the data 
needs to be put in is only available from the idt_match table. So I have 
to do at least one join with idt_match. But I will experiment further 
with ways to perform queries, I just stopped doing that because I saw 
these problems with scalability.

> I don't have much experience at data partitioning (well, I do, but the
> experience is with partitioning in Perl with terabytes of flat files,
> not in PG :) ) but I think that once you have your partitioning keys
> you want to apply them the same way up and down the data set.

I'm not sure what you mean by "the same way up and down the data set".

Cheers,
 	S

*****************************************************
Sergey E. Koposov, PhD, Research Associate
Institute of Astronomy, University of Cambridge
Madingley road, CB3 0HA, Cambridge, UK
Tel: +44-1223-337-551 Web: http://www.ast.cam.ac.uk/~koposov/