Horton

Re: Parallel bitmap heap scan

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Dilip Kumar <dilipbalaut@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2016-10-17T20:15:58Z
Lists: pgsql-hackers 
Hi,

On 2016-10-07 11:46:40 +0530, Dilip Kumar wrote:
> Brief design idea:
> -----------------------
> #1. Shared TIDBitmap creation and initialization
>   First worker to see the state as parallel bitmap info as PBM_INITIAL
> become leader and set the state to PBM_INPROGRESS All other   workers
> see the state as PBM_INPROGRESS will wait for leader to complete the
> TIDBitmap.
> 
> #2 At this level TIDBitmap is ready and all workers are awake.
> 
> #3. Bitmap processing (Iterate and process the pages).
> In this phase each worker will iterate over page and chunk array and
> select heap pages one by one. If prefetch is enable then there will be
> two iterator. Since multiple worker are iterating over same page and
> chunk array we need to have a shared iterator, so we grab a spin lock
> and iterate within a lock, so that each worker get and different page
> to process.

I don't quite understand why the bitmap has to be parallel at all. As
far as I understand your approach as described here, the only thing that
needs to be shared are the iteration arrays.  Since they never need to
be resized and such, it seems to make a lot more sense to just add an
API to share those, instead of the whole underlying hash.

Greetings,

Andres Freund

Commits

  1. Pass DSA_ALLOC_HUGE when allocating a shared TIDBitmap.

  2. Support parallel bitmap heap scans.

  3. tidbitmap: Support shared iteration.

  4. Shut down Gather's children before shutting down Gather itself.

  5. Revise the way the element allocator for a simplehash is specified.

  6. Refactor bitmap heap scan estimation of heap pages fetched.

  7. Use more efficient hashtable for tidbitmap.c to speed up bitmap scans.