Re: Hash-based MCV matching for large IN-lists

Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>

From: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
To: Zsolt Parragi <zsolt.parragi@percona.com>
Cc: David Geier <geidav.pg@gmail.com>, Chengpeng Yan <chengpeng_yan@outlook.com>, Tatsuya Kawata <kawatatatsuya0913@gmail.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>
Date: 2026-03-10T14:55:57Z
Lists: pgsql-hackers

Attachments

In the thread discussing ALL semantics and NULL [0], the question was 
raised about adding a new regression test that checks selectivity 
estimation. If the change gets committed, it would make sense to add 
tests for this case as well.

Regarding the idea of optimizing the loop when all per-element 
selectivities are the same: I ran some quick tests to see how much the 
change in the v7-0002 patch affects planning time. Even without that 
patch, iterating over an array with 50k elements takes about 30 ms.

```
CREATE TABLE t (val bytea PRIMARY KEY);
INSERT INTO t SELECT int4send(i) FROM generate_series(1,50000) AS i;
ANALYZE t;
SELECT n_distinct FROM pg_stats WHERE tablename = 't';
  n_distinct
------------
          -1
(1 row)

SELECT string_agg(format('int4send(%s)', i), ',') FROM 
generate_series(1,50000) AS i \gset
EXPLAIN (SUMMARY) SELECT * FROM t WHERE val = ANY 
(ARRAY[:string_agg]::bytea[]);
..........
  Planning Time: 32.816 ms
(3 rows)
```

Given that, I don't see much benefit in adding additional logic here 
just to avoid the loop. It would likely introduce extra code complexity 
without a manful gain. If there is interest in optimization this case 
further, I can revisit it and add the additional patch.

The patch v8 can still be reviewed as-is, and if the selectivity 
regression test gets committed [0], I will add corresponding tests for 
this change as well.

[0]: 
https://www.postgresql.org/message-id/390a46f3-dbc4-4dc1-b49d-5cc61dd36026%40tantorlabs.com

-- 
Best regards,
Ilia Evdokimov,
Tantor Labs LLC,
https://tantorlabs.com/

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Short-circuit row estimation in NOT IN containing NULL consts