Re: Proposal: Improve bitmap costing for lossy pages

Alexander Kuzmenkov <a.kuzmenkov@postgrespro.ru>

From: Alexander Kuzmenkov <a.kuzmenkov@postgrespro.ru>
To: Dilip Kumar <dilipbalaut@gmail.com>, Robert Haas <robertmhaas@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2017-10-06T12:38:52Z
Lists: pgsql-hackers
> Analysis: The estimated value of the lossy_pages is way higher than
> its actual value and reason is that the total_pages calculated by the
> "Mackert and Lohman formula" is not correct.

I think the problem might be that the total_pages includes cache effects 
and rescans. For bitmap entries we should use something like relation 
pages * selectivity.

Meanwhile, I ran TPC-H briefly with the v3 patch: scale factor 25, 2 
workers, SSD storage.
It shows significant improvement on 4MB work_mem and no change on 2GB.

Here are the results (execution time in seconds, average of 5 runs):
work_mem    4MB                2GB
Query     master    patch    master    patch
4        179        174        168        167
5        190        168        155        156
6        280        87        227        229
8        197        114        179        172
10        269        227        190        192
14        110        108        106        105

-- 
Alexander Kuzmenkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company



Commits

  1. Account for the effect of lossy pages when costing bitmap scans.