Re: Use merge-based matching for MCVs in eqjoinsel
Tom Lane <tgl@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
To: David Geier <geidav.pg@gmail.com>
Cc: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>,
pgsql-hackers@lists.postgresql.org
Date: 2025-11-19T02:19:46Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Speed up eqjoinsel() with lots of MCV entries.
- 057012b205a0 19 (unreleased) landed
Attachments
- eqseltest1-log.sql (text/plain)
- eqseltest2.sql (text/plain)
I wrote: > Thinking a bit harder, we are comparing these costs: > [ theoretical arguments trimmed ] I spent some effort on actually measuring timings of the v6 patch, and concluded that this is all splitting hairs that we don't need to split. The actual crossover between hash-loses and hash-wins is more than what my theoretical argument suggested, but still probably less than 100 MCVs on each side. I think we should go with (sslot1.nvalues + sslot2.nvalues) >= 200 and call it good. To arrive at this result, I built the v6 patchset with EQJOINSEL_MCV_HASH_THRESHOLD changed to either 0 (to force hashing) or 1000000 (to prevent it). I then ran the attached scripts with different values of "nstats" and collected timings from the postmaster log output produced by the 0001 patch. The scripts are designed to test both the cheap-comparisons scenario (integer columns) and the expensive-comparisons scenario (text columns with a case-insensitive ICU collation). My motivation for splitting them into a setup and a test step was to allow the tests to be run repeatedly against the same underlying data. (Although I soon realized that because VACUUM ANALYZE takes a random sample each time, the stats we're working from aren't totally the same each time anyway.) Also you'll notice that the test data is based on log(random()), which I did to roughly approximate a zipfian distribution. If you remove the log() call you'll get a flat distribution instead, but it didn't seem to change the conclusions much. regards, tom lane