Performance issue in foreign-key-aware join estimation
Tom Lane <tgl@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
To: pgsql-hackers@lists.postgresql.org
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Date: 2018-12-20T17:44:17Z
Lists: pgsql-hackers
Attachments
- schema.sql (text/plain)
- query.sql (text/plain)
In connection with David Rowley's proposal to change bitmapset.c to use 64-bit words, I dug out an old test case I had for a complex-to-plan query (attached). Andres Freund posted this to the lists perhaps ten years ago, though I can't locate the original posting right now. I was distressed to discover via perf that 69% of the runtime of this test now goes into match_eclasses_to_foreign_key_col(). That seems clearly unacceptable. There aren't an unreasonable number of foreign key constraints in the underlying schema --- mostly one per table, and there's only one table with as many as 3. However, poking around in the planner data structures, it turns out there are: 888 base relations 1005 EquivalenceClasses 167815 fkey_list entries initially 690 fkey_list entries after match_foreign_keys_to_quals trims them So the reason match_eclasses_to_foreign_key_col is so dominant in the runtime is it's invoked 167815 times and has to scan a thousand EquivalenceClasses (unsuccessfully) on most of those calls. How did the fkey_list get that big? I think the issue is that the query touches the same tables many many times (888 baserels, but there are only 20 distinct tables in the schema) and we get an O(N^2) growth in the apparent number of FKs. Clearly, we ought to rethink that data structure. I'm not sure offhand how to make it better, but this is pretty awful. Perhaps there'd also be some use in having better indexing for the EquivalenceClass list, but again I'm not sure what that'd look like. regards, tom lane
Commits
-
Adjust overly strict Assert
- e1a0f6a98306 13.0 landed
-
Speed up finding EquivalenceClasses for a given set of rels
- 3373c7155350 13.0 landed
-
Speed up match_eclasses_to_foreign_key_col() when there are many ECs.
- fa86238f1eac 12.0 landed
- e22bfe94e4df 11.3 landed
- affee8b14c68 10.8 landed
- 09c9dd39efea 9.6.13 landed