Re: Parallel tuplesort (for parallel B-Tree index creation)

Rushabh Lathia <rushabh.lathia@gmail.com>

From: Rushabh Lathia <rushabh.lathia@gmail.com>

To: Peter Geoghegan <pg@bowt.ie>

Cc: Thomas Munro <thomas.munro@enterprisedb.com>, Robert Haas <robertmhaas@gmail.com>, Heikki Linnakangas <hlinnaka@iki.fi>, Pg Hackers <pgsql-hackers@postgresql.org>, Corey Huinker <corey.huinker@gmail.com>

Date: 2017-10-26T11:22:16Z

Lists: pgsql-hackers

Attachments

0001-Add-parallel-B-tree-index-build-sorting_v12_rebase.patch (text/x-patch) patch v12-0001

Attaching the re based patch according to the v22 parallel-hash patch sets

Thanks

On Tue, Oct 10, 2017 at 2:53 PM, Rushabh Lathia <rushabh.lathia@gmail.com>
wrote:

>
>
> On Sat, Sep 30, 2017 at 5:06 AM, Peter Geoghegan <pg@bowt.ie> wrote:
>
>> On Wed, Sep 20, 2017 at 2:32 AM, Rushabh Lathia
>> <rushabh.lathia@gmail.com> wrote:
>> > Yes, I haven't touched the randomAccess part yet. My initial goal was
>> > to incorporate the BufFileSet api's here.
>>
>> This is going to need a rebase, due to the commit today to remove
>> replacement selection sort. That much should be easy.
>>
>>
> Sorry for delay, here is rebase version of patch.
>
>
>> > Sorry, I didn't get this part. Are you talking about the your patch
>> changes
>> > into OpenTemporaryFileInTablespace(),  BufFileUnify() and other changes
>> > related to ltsUnify() ?  If that's the case, I don't think it require
>> with
>> > the
>> > BufFileSet. Correct me if I am wrong here.
>>
>> I thought that you'd have multiple BufFiles, which would be
>> multiplexed (much like a single BufFile itself mutiplexes 1GB
>> segments), so that logtape.c could still recycle space in the
>> randomAccess case. I guess that that's not a goal now.
>>
>>
> Hmm okay.
>
>
>> > To be frank its too early for me to comment anything in this area.  I
>> need
>> > to study this more closely. As an initial goal I was just focused on
>> > understanding the current implementation of the patch and incorporate
>> > the BufFileSet APIs.
>>
>> Fair enough.
>>
>>
> Thanks,
>
> --
> Rushabh Lathia
> www.EnterpriseDB.com
>



-- 
Rushabh Lathia

Commits

Support parallel btree index builds.
- 9da0cc35284b 11.0 landed
Report an ERROR if a parallel worker fails to start properly.
- 2badb5afb89c 11.0 cited
Transfer state pertaining to pending REINDEX operations to workers.
- 29d58fd3adae 11.0 landed
Add a barrier primitive for synchronizing backends.
- 1145acc70deb 11.0 cited
Allow DML commands that create tables to use parallel query.
- e9baa5e9fa14 11.0 cited
Refactor GetOldestXmin() to use flags
- af4b1a0869bd 10.0 cited
Fix regression in parallel planning against inheritance tables.
- 2609e91fcf9d 10.0 cited
Don't create "holes" in BufFiles, in the new logtape code.
- 7ac4a389a7db 10.0 cited
Simplify the code for logical tape read buffers.
- b75f467b6eec 10.0 cited
Fix excessive memory consumption in the new sort pre-reading code.
- b56fb691b003 10.0 cited
Implement binary heap replace-top operation in a smarter way.
- 24598337c8d2 10.0 cited
Cosmetic code cleanup in commands/extension.c.
- 25794e841e5b 10.0 cited
Speed up planner's scanning for parallel-query hazards.
- da1c91631e35 10.0 cited
Read from the same worker repeatedly until it returns no tuple.
- bc7fcab5e36b 9.6.0 cited
Improve tuplesort.c to support variable merge order. The original coding
- df700e6b4019 8.2.0 cited