Re: Use merge-based matching for MCVs in eqjoinsel

Tom Lane <tgl@sss.pgh.pa.us>

From: Tom Lane <tgl@sss.pgh.pa.us>
To: David Geier <geidav.pg@gmail.com>
Cc: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>, pgsql-hackers@lists.postgresql.org
Date: 2025-11-19T02:19:46Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Speed up eqjoinsel() with lots of MCV entries.

Attachments

I wrote:
> Thinking a bit harder, we are comparing these costs:
> [ theoretical arguments trimmed ]

I spent some effort on actually measuring timings of the v6 patch,
and concluded that this is all splitting hairs that we don't need
to split.  The actual crossover between hash-loses and hash-wins
is more than what my theoretical argument suggested, but still
probably less than 100 MCVs on each side.  I think we should go with

	(sslot1.nvalues + sslot2.nvalues) >= 200

and call it good.

To arrive at this result, I built the v6 patchset with
EQJOINSEL_MCV_HASH_THRESHOLD changed to either 0 (to force hashing)
or 1000000 (to prevent it).  I then ran the attached scripts with
different values of "nstats" and collected timings from the postmaster
log output produced by the 0001 patch.

The scripts are designed to test both the cheap-comparisons scenario
(integer columns) and the expensive-comparisons scenario (text columns
with a case-insensitive ICU collation).  My motivation for splitting
them into a setup and a test step was to allow the tests to be run
repeatedly against the same underlying data.  (Although I soon realized
that because VACUUM ANALYZE takes a random sample each time, the stats
we're working from aren't totally the same each time anyway.)  Also
you'll notice that the test data is based on log(random()), which
I did to roughly approximate a zipfian distribution.  If you remove
the log() call you'll get a flat distribution instead, but it didn't
seem to change the conclusions much.

			regards, tom lane