Horton

Re: WIP: a way forward on bootstrap data

John Naylor <jcnaylor@gmail.com>

View thread View on postgr.es
From: John Naylor <jcnaylor@gmail.com>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: David Fetter <david@fetter.org>, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2017-12-28T11:26:23Z
Lists: pgsql-hackers

Attachments

I  wrote:

> On 12/22/17, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>> Maybe a generated .h file containing defines for OIDs from all catalogs
>> is okay.  Of course, not all symbols are to be listed -- we should have
>> a special marker in the data lines for those that are.  Maybe something
>> like this
>>
>> { oid => '403', descr => 'b-tree index access method',
>>   amname => 'btree', amhandler => 'bthandler', amtype => 'i',
>>   cpp_symbol => 'BTREE_AM_OID' },
>>
>> (where 'cpp_symbol' would be skipped by genbki explicitly).
>
> The last part makes sense and would be a fairly mechanical change. I'm
> not sure of the best way to include those generated symbols back in
> the code again, though. I think a single file might not be desirable
> because of namespace pollution.
[snip]

I've attached version 5 (rebased onto be2343221fb), which removes OID
#define symbols from the headers and stores them with the records they
refer to.

I went ahead with your suggestion to try a single generated header. I
believe there is no chance of namespace pollution since the symbols
already have a nomenclature that reflects what catalog they belong to.
This required some additional Makefile changes, since some code
outside the backend needs certain OID symbols to be visible. There are
probably bugs, but I wanted to share the initial design. The MSVC
changes are untested. In addition, FindDefinedSymbol() now doesn't
work for catalog headers, so I added a new function to search within
the data.

On the plus side, there is now a mechanism to generate pg_type OID
symbols, and ECPG's knowledge of types is now maintained
automatically.

Since I had to rebase over recent additions to SP-GiST opclasses
anyway, I restructured the patches to have a clean separation between
data migration and data compaction. This makes the patches easier to
follow.

The pg_proc defaults have been tweaked slightly to match those
suggested by Andrew Dunstan [1].

There are now human-readable entries for oprcom and oprnegate in
pg_operator.dat.

Finally, assorted cosmetic improvements and README/comment editing.

[1] https://www.postgresql.org/message-id/b76d153a-33d7-7827-746c-1109f7bf529d%40dunslane.net
--

-John Naylor

Commits

  1. Clarify handling of special-case values in bootstrap catalog data.

  2. Replace our traditional initial-catalog-data format with a better design.

  3. Faster partition pruning

  4. Minor cleanup in genbki.pl.

  5. Trivial adjustments in preparation for bootstrap data conversion.

  6. Remove hard-coded schema knowledge about pg_attribute from genbki.pl

  7. Minor edits to catalog files and scripts

  8. Hide most variable-length fields from Form_pg_* structs