Re: WIP: a way forward on bootstrap data

John Naylor <jcnaylor@gmail.com>

View thread View on postgr.es

From: John Naylor <jcnaylor@gmail.com>

To: David Fetter <david@fetter.org>

Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>

Date: 2017-12-21T18:32:22Z

Lists: pgsql-hackers

Attachments

v4-0001-Fix-typos-and-oversights-in-catalog-headers.patch (text/x-patch) patch v4-0001
v4-0002-Cleanup-distprep-scripts.patch (text/x-patch) patch v4-0002
v4-0003-Remove-hard-coded-schema-knowledge-about-pg_attri.patch (text/x-patch) patch v4-0003
v4-0004-Data-conversion-infrastructure.patch (text/x-patch) patch v4-0004
v4-0005-Mechanical-data-conversion.patch.tar.gz (application/x-gzip)
v4-0006-Hand-edits-of-data-files.patch (text/x-patch) patch v4-0006
v4-0007-Update-catalog-scripts-to-read-data-files.patch (text/x-patch) patch v4-0007
v4-0008-Remove-data-from-catalog-headers.patch.tar.gz (application/x-gzip)
v4-0009-Replace-oids-with-human-readable-names.patch.tar.gz (application/x-gzip)
v4-0010-Type-aliases-for-oid-lookups.patch (text/x-patch) patch v4-0010
v4-0011-Reduce-indentation-level.patch (text/x-patch) patch v4-0011
v4-0012-Move-toast-index-macros-to-the-pg_-catalog-header.patch (text/x-patch) patch v4-0012

On 12/20/17, David Fetter <david@fetter.org> wrote:
> Thanks for your hard work on this.  It'll really make developing this
> part of the code a lot more pleasant.

I hope so, thanks.

> Unfortunately, it no longer applies to master.  Can we get a rebase?

Hmm, it still applied for me, except when I forgot to gunzip the
larger patches. In any case, I've attached version 4 which contains
some recent improvements. It was rebased against master as of
6719b238e8f0ea. If it doesn't apply for you please let me know the
details.
--

New in this patch set:
-Remove code duplication and improve modularity in data parsing
-Update README at appropriate points
-Shift some hunks around to make patches more focused and readable
-Comment and commit message editing
-Patch 11 reduces indentation
-Patch 12 moves all the toast/index commands into the individual
catalog headers and simplifies some #includes (Note: I failed to shave
the yak of selinux, so the #include changes for contrib/sepgsql are
untested)

At this point, I think it's no longer a WIP, and will only make
further changes based on review or if I find a mistake.

Left for future projects:
-SQL generation for querying source bootstrap data
-Generating pg_type #defines

-John Naylor

Commits

Clarify handling of special-case values in bootstrap catalog data.
- 45c6d75f8cd2 11.0 landed
Replace our traditional initial-catalog-data format with a better design.
- 372728b0d495 11.0 landed
Faster partition pruning
- 9fdb675fc5d2 11.0 cited
Minor cleanup in genbki.pl.
- 8d90b4d01a61 11.0 landed
Trivial adjustments in preparation for bootstrap data conversion.
- a351679c806e 11.0 landed
Remove hard-coded schema knowledge about pg_attribute from genbki.pl
- 49c784ece766 11.0 landed
Minor edits to catalog files and scripts
- 9373baa0f764 11.0 landed
Hide most variable-length fields from Form_pg_* structs
- 8137f2c32322 9.2.0 cited