Re: Parallel bitmap heap scan
Amit Kapila <amit.kapila16@gmail.com>
From: Amit Kapila <amit.kapila16@gmail.com>
To: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>,
Haribabu Kommi <kommi.haribabu@gmail.com>, Rafia Sabih <rafia.sabih@enterprisedb.com>,
tushar <tushar.ahuja@enterprisedb.com>, Amit Khandekar <amitdkhan.pg@gmail.com>, Andres Freund <andres@anarazel.de>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2017-02-19T14:06:45Z
Lists: pgsql-hackers
On Sun, Feb 19, 2017 at 7:18 PM, Dilip Kumar <dilipbalaut@gmail.com> wrote: > On Sat, Feb 18, 2017 at 10:45 PM, Dilip Kumar <dilipbalaut@gmail.com> wrote: >> in 0002: >> - Improved comments. >> - Code refactoring in BitmapHeapNext. >> - Removed local tbm creation in BitmapHeapNext : as per new tidbitmap >> it's of no use. > > I have observed one problem with 0002 and I though of sharing that > before fixing the same because we might have encountered the same > problem in some other patches i.e parallel shared hash and there might > be already a way to handle that. > > The problem is that In ExecEndBitmapHeapScan we do tbm_free. Wherein, > it frees local pagetable memory and the shared memory using callback > routine and if other than the Backend (actual backend for the client > which is executing gather node) node any other worker become the > leader (worker which is responsible for creating the shared TBM) then > it will finish it work and free the shared memory by calling > ExecEndBitmapHeapScan, and it's possible that other worker are still > operating on the shared memory. > So far I never observed this issue in our test may be because either > Backend become the leader or by the time leader (non backend) free the > TBM others also finishes there work. > > Solution: > - One solution to this problem can be that leader should not complete > the scan unless all other worker have finished the work. For Parallel Seq Scan, we do wait for parallel workers to finish before freeing the shared memory. Why the case is different here? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
Commits
-
Pass DSA_ALLOC_HUGE when allocating a shared TIDBitmap.
- a171f01501e1 10.0 landed
-
Support parallel bitmap heap scans.
- f35742ccb7aa 10.0 landed
-
tidbitmap: Support shared iteration.
- 98e6e89040a0 10.0 landed
-
Shut down Gather's children before shutting down Gather itself.
- acf555bc53ac 10.0 landed
-
Revise the way the element allocator for a simplehash is specified.
- c3c4f6e1740b 10.0 cited
-
Refactor bitmap heap scan estimation of heap pages fetched.
- da08a6598973 10.0 landed
-
Use more efficient hashtable for tidbitmap.c to speed up bitmap scans.
- 75ae538bc316 10.0 cited