Re: WIP: a way forward on bootstrap data
John Naylor <jcnaylor@gmail.com>
From: John Naylor <jcnaylor@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2018-03-09T08:32:59Z
Lists: pgsql-hackers
Attachments
- v9-convert_header2dat.pl (application/x-perl)
- v9-0001-Create-data-conversion-infrastructure.patch (text/x-patch) patch v9-0001
- v9-0002-Hand-edits-of-data-files.patch (text/x-patch) patch v9-0002
- v9-0003-Update-catalog-scripts-to-read-data-files.patch (text/x-patch) patch v9-0003
- v9-0004-Clean-up-header-files-and-update-comments.patch (text/x-patch) patch v9-0004
- v9-0005-Remove-symbols-from-catalog-headers.patch (text/x-patch) patch v9-0005
- v9-apply-bootstrap-data-patches.sh (application/x-sh)
On 3/4/18, Tom Lane <tgl@sss.pgh.pa.us> wrote: > John Naylor <jcnaylor@gmail.com> writes: >> On 3/3/18, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> * In 0006, I'm not very pleased with the introduction of >>> "Makefile.headers". > >> I wasn't happy with it either, but I couldn't get it to build >> otherwise. The sticking point was the symlinks in >> $(builddir)/src/include/catalog. $(MAKE) -C catalog doesn't handle >> that. The makefile in /src/common relies on the backend makefile to >> know what to invoke for a given header. IIRC, relpath.c includes >> pg_tablespace.h, which now requires pg_tablespace_d.h to be built. > > I'm not following. AFAICS, what you put in src/common/Makefile was just > > +.PHONY: generated-headers > + > +generated-headers: > + $(MAKE) -C ../backend generated-headers > > which doesn't appear to care whether backend/Makefile knows anything > about specific generated headers or not. I think all we need to do > is consider that the *_d.h files ought to be built as another consequence > of invoking the generated-headers target. > > BTW, there's already a submake-generated-headers target in > Makefile.global, which you should use in preference to rolling your own. I've attached version 9, whose biggest change is to address the above points of review. I pushed all of the catalog header build logic into catalog Makefile to avoid creating a separate symbol file. This involved putting the distprep logic there as well. Enough of the structure changed that one or two names didn't make sense anymore, so I changed them. As suggested, the conversion script is now part of the patchset and not committed to the repo. To run the conversion, save everything to a directory and update the dir vars at the top of apply-bootstrap-data-patches.sh accordingly. A couple things to note that I didn't do: -With all the new generated headers, the message "Writing ..." is now quite verbose. It might be worth changing that. -I'm not sure if I need to change anything involving "make install". -I haven't tested the MSVC changes. -I didn't change any clients to actually use the new headers directly. That might be too ambitious for this cycle anyway. While this goes through review, I'll get a head start rebasing the human readable OIDs and data compaction patches. -John Naylor
Commits
-
Clarify handling of special-case values in bootstrap catalog data.
- 45c6d75f8cd2 11.0 landed
-
Replace our traditional initial-catalog-data format with a better design.
- 372728b0d495 11.0 landed
-
Faster partition pruning
- 9fdb675fc5d2 11.0 cited
-
Minor cleanup in genbki.pl.
- 8d90b4d01a61 11.0 landed
-
Trivial adjustments in preparation for bootstrap data conversion.
- a351679c806e 11.0 landed
-
Remove hard-coded schema knowledge about pg_attribute from genbki.pl
- 49c784ece766 11.0 landed
-
Minor edits to catalog files and scripts
- 9373baa0f764 11.0 landed
-
Hide most variable-length fields from Form_pg_* structs
- 8137f2c32322 9.2.0 cited