Horton

Re: Explain buffers wrong counter with parallel plans

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>, Adrien Nayrat <adrien.nayrat@anayrat.info>, Andres Freund <andres@anarazel.de>, PostgreSQL mailing lists <pgsql-hackers@postgresql.org>
Date: 2018-07-06T13:44:00Z
Lists: pgsql-hackers 
On Tue, Jul 3, 2018 at 4:18 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Mon, Jul 2, 2018 at 6:02 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>>
>
>> To fix the problem with Limit that you mention, we could just modify
>> nodeLimit.c so that when the state is changed from LIMIT_INWINDOW to
>> LIMIT_WINDOWEND, we also call ExecShutdownNode on the child plan.
>>
>
> It should work.
>

I have tried this idea, but it doesn't completely solve the problem.
The problem is that nodes below LIMIT won't get a chance to accumulate
the stats as they won't be able to call InstrStopNode.  So the result
will be something like below:

postgres=# explain (analyze,buffers,timing off,costs off) select *
from t1 limit 50000;
                           QUERY PLAN
-----------------------------------------------------------------
 Limit (actual rows=50000 loops=1)
   Buffers: shared hit=6 read=224
   ->  Gather (actual rows=50000 loops=1)
         Workers Planned: 2
         Workers Launched: 2
         Buffers: shared hit=1 read=63
         ->  Parallel Seq Scan on t1 (actual rows=17213 loops=3)
               Buffers: shared hit=6 read=224
 Planning Time: 0.105 ms
 Execution Time: 1363068.675 ms
(10 rows)

In this plan, you can notice that stats (Buffers:) at Parallel Seq
Scan and Limit are same, but Gather node shows different stats.  One
idea could be that in ExecShutdownNode, somehow, we allow the nodes to
count the stats, maybe by calling InstrStartNode/InstrStopNode, but
not sure if that is the best way to fix it.

Thoughts?

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Commits

  1. Adjust comment atop ExecShutdownNode.

  2. Prohibit shutting down resources if there is a possibility of back up.

  3. Fix buffer usage stats for parallel nodes.

  4. Match the buffer usage tracking for leader and worker backends.

  5. Improve division of labor between execParallel.c and nodeGather[Merge].c.