Re: weird hash plan cost, starting with pg10
Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
From: Konstantin Knizhnik <k.knizhnik@postgrespro.ru>
To: Richard Guo <guofenglinux@gmail.com>,
Thomas Munro <thomas.munro@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Alvaro Herrera <alvherre@2ndquadrant.com>,
Pg Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2020-03-25T14:29:59Z
Lists: pgsql-hackers
Attachments
- hash_join_instrumentation.patch (text/x-patch) patch
On 25.03.2020 13:36, Richard Guo wrote: > > On Tue, Mar 24, 2020 at 3:36 PM Richard Guo <guofenglinux@gmail.com > <mailto:guofenglinux@gmail.com>> wrote: > > On Tue, Mar 24, 2020 at 11:05 AM Thomas Munro > <thomas.munro@gmail.com <mailto:thomas.munro@gmail.com>> wrote: > > > I think there might be a case like this: > > * ExecRescanHashJoin() decides it can't reuse the hash table for a > rescan, so it calls ExecHashTableDestroy(), clears HashJoinState's > hj_HashTable and sets hj_JoinState to HJ_BUILD_HASHTABLE > * the HashState node still has a reference to the pfree'd > HashJoinTable! > * HJ_BUILD_HASHTABLE case reaches the empty-outer optimisation > case so > it doesn't bother to build a new hash table > * EXPLAIN examines the HashState's pointer to a freed > HashJoinTable struct > > > Yes, debugging with gdb shows this is exactly what happens. > > > According to the scenario above, here is a recipe that reproduces this > issue. > > -- recipe start > create table a(i int, j int); > create table b(i int, j int); > create table c(i int, j int); > > insert into a select 3,3; > insert into a select 2,2; > insert into a select 1,1; > > insert into b select 3,3; > > insert into c select 0,0; > > analyze a; > analyze b; > analyze c; > > set enable_nestloop to off; > set enable_mergejoin to off; > > explain analyze > select exists(select * from b join c on a.i > c.i and a.i = b.i and > b.j = c.j) from a; > -- recipe end > > I tried this recipe on different PostgreSQL versions, starting from > current master and going backwards. I was able to reproduce this issue > on all versions above 8.4. In 8.4 version, we do not output information > on hash buckets/batches. But manual inspection with gdb shows in 8.4 we > also have the dangling pointer for HashState->hashtable. I didn't check > versions below 8.4 though. > > Thanks > Richard I can propose the following patch for the problem. -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
Commits
-
Make EXPLAIN report maximum hashtable usage across multiple rescans.
- 969f9d0b4ba5 13.0 landed
-
Clear dangling pointer to avoid bogus EXPLAIN printout in a corner case.
- f3d06e524073 11.8 landed
- 8ffb86644530 12.3 landed
- 5c27bce7f39d 13.0 landed
- 30ce86367e3e 9.5.22 landed
- 242ca479fea1 9.6.18 landed
- 1e6bb6125f33 10.13 landed
-
psql: Add tab completion for logical replication
- 6f236e1eb8c7 10.0 cited
-
Make the upper part of the planner work by generating and comparing Paths.
- 3fc6e2d7f5b6 9.6.0 cited