List of Bitmapset (was Re: ExecRTCheckPerms() and many prunable partitions)

Tom Lane <tgl@sss.pgh.pa.us>

From: Tom Lane <tgl@sss.pgh.pa.us>
To: Amit Langote <amitlangote09@gmail.com>
Cc: Richard Guo <guofenglinux@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, David Rowley <dgrowleyml@gmail.com>, Greg Stark <stark@mit.edu>, Julien Rouhaud <rjuju123@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-11-14T14:57:18Z
Lists: pgsql-hackers

Attachments

[ I'm intentionally forking this off as a new thread, so as to
not confuse the cfbot about what's the live patchset on the
ExecRTCheckPerms thread. ]

Amit Langote <amitlangote09@gmail.com> writes:
> On Sat, Nov 12, 2022 at 1:46 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> The main thing I was wondering about in connection with that
>> was whether to assume that there could be other future applications
>> of the logic to perform multi-bitmapset union, intersection,
>> etc.  If so, then I'd be inclined to choose different naming and
>> put those functions in or near to bitmapset.c.  It doesn't look
>> like Amit's code needs anything like that, but maybe somebody
>> has an idea about other applications?

> Yes, simple storage of multiple Bitmapsets in a List somewhere in a
> parse/plan tree sounded like that would have wider enough use to add
> proper node support for.   Assuming you mean trying to generalize
> VarAttnoSet in your patch 0004 posted at [2], I wonder if you want to
> somehow make its indexability by varno / RT index a part of the
> interface of the generic code you're thinking for it?

For discussion's sake, here's my current version of that 0004 patch,
rewritten to use list-of-bitmapset as the data structure.  (This
could actually be pulled out of the outer-join-vars patchset and
committed independently, just as a minor performance improvement.
It doesn't quite apply cleanly to HEAD, but pretty close.)

As it stands, the new functions are still in util/clauses.c, but
if we think they could be of general use it'd make sense to move them
either to nodes/bitmapset.c or to some new file under backend/nodes.

Some other thoughts:

* The multi_bms prefix is a bit wordy, so I was thinking of shortening
the function names to mbms_xxx.  Maybe that's too brief.

* This is a pretty short list of functions so far.  I'm not eager
to build out a bunch of dead code though.  Is it OK to leave it
with just this much functionality until someone needs more?

* I'm a little hesitant about whether the API actually should be
List-of-Bitmapset, or some dedicated struct as I had in the previous
version of 0004.  This version is way less invasive in prepjointree.c
than that was, but the reason is there's ambiguity about what the
forced_null_vars Lists actually contain, which feels error-prone.

Comments?

			regards, tom lane

Commits

  1. Invent "multibitmapsets", and use them to speed up antijoin detection.