Re: reducing the footprint of ScanKeyword (was Re: Large writable variables)

Tom Lane <tgl@sss.pgh.pa.us>

From: Tom Lane <tgl@sss.pgh.pa.us>
To: Joerg Sonnenberger <joerg@bec.de>
Cc: David Rowley <david.rowley@2ndquadrant.com>, John Naylor <jcnaylor@gmail.com>, Andres Freund <andres@anarazel.de>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2019-01-06T20:24:55Z
Lists: pgsql-hackers
Joerg Sonnenberger <joerg@bec.de> writes:
> On Sun, Jan 06, 2019 at 02:29:05PM -0500, Tom Lane wrote:
>> So we probably can't have inlined hashing code --- I imagine the
>> hash generator needs the flexibility to pick different values of
>> those multipliers.

> Right now, only the initial values are randomized. Picking a different
> set of hash functions is possible, but someone that should be done only
> if there is an actual need. That was what I meant with stronger mixing
> might be necessary for "annoying" keyword additions.

Hmm.  I'm still leaning towards using generated, out-of-line hash
functions though, because then we could have a generator switch
indicating whether to apply the |0x20 case coercion or not.
(I realize that we could blow off that consideration and use a
case-insensitive hash function all the time, but it seems cleaner
to me not to make assumptions about how variable the hash function
parameters will need to be.)

> There are two ways for dealing with it:
> (1) Have one big hash table with all the various keywords and a class
> mask stored. If there is enough overlap between the keyword tables, it
> can significantly reduce the amount of space needed. In terms of code
> complexity, it adds one class check at the end, i.e. a bitmap test.

No, this would be a bad idea IMO, because it makes the core, plpgsql,
and ecpg keyword sets all interdependent.  If you add a keyword to any
one of those and forget to rebuild the other components, you got trouble.
Maybe we could make that reliable, but I don't think it's worth fooling
with for hypothetical benefit.  Also, it'd make the net space usage more
not less, because each of those executables/shlibs would contain copies
of all the keywords for the other ones' needs.

			regards, tom lane


Commits

  1. Use perfect hashing, instead of binary search, for keyword lookup.

  2. Reduce the size of the fmgr_builtin_oid_index[] array.

  3. Replace the data structure used for keyword lookup.