Horton

Thread

Commits

  1. docs: meson: Change what 'docs' target builds

  2. meson: Add 'world' target

  3. meson: Document build targets, add 'help' target

  4. meson: docs: Add {html,man} targets, rename install-doc-*

  5. docs: Document --with-selinux/-Dselinux options centrally

  6. meson: Change default of 'selinux' feature option to auto

  7. meson: Fix doc installation path computation

  8. meson: Install missing example files

  9. meson: docs: Install all manpages, not just ones in man1

  10. meson: docs: Allow configuring simple/website style

  11. docs: html: load stylesheet via custom.css.source

  12. docs: html: copy images to output as part of xslt build

  13. meson: docs: Preparatory cleanups

  14. docs: Remove support for 'htmlhelp' format

  15. meson: add install-{docs,doc-html,doc-man} targets

  16. meson: add install-{quiet, world} targets

  17. meson: make install_test_files more generic, rename to install_files

  18. meson: rename html_help target to htmlhelp

  1. meson documentation build open issues

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-03-15T07:14:09Z

    I have identified several open issues with the documentation build under 
Meson (approximately in priority order):

1. Image files are not handled at all, so they don't show up in the 
final product.

2. Defaults to website stylesheet, no way to configure.  This should be 
adjusted to match the make build.

3. The various build targets and their combinations are mismatching and 
incomplete.  For example:

Top-level GNUmakefile has these targets:

- docs (builds html and man)
- html
- man

(Those are the formats that are part of a distribution build.)

doc/src/sgml/Makefile has these documented targets:

- default target is html
- all (builds html and man, maps to top-level "docs")
- html
- man
- postgres-A4.pdf
- postgres-US.pdf
- check

as well as (undocumented):

- htmlhelp
- postgres.html
- postgres.txt
- epub
- postgres.epub
- postgres.info

meson has the following documented targets:

- docs (builds only html)
- alldocs (builds all formats, including obscure ones)

as well as the following undocumented targets:

- html
- man
- html_help [sic]
- postgres-A4.pdf
- postgres-US.pdf
- postgres.epub

- [info is not implemented at all]
- [didn't find an equivalent of check]

As you can see, this is all over the place.  I'd like to arrive at some 
consistency across all build systems for handling each tier of 
documentation formats, in terms of what is documented, what the targets 
are named, and how they are grouped.

4. There doesn't appear to be a way to install the documentation.
(There are also some open questions in the top-level meson.build about
the installation directories, but I suppose if we can't install them
then exactly where to install them hasn't been thought about too
much.)

5. There doesn't appear to be an equivalent of "make world" and "make
install-world" that includes documentation builds.

  2. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-03-16T03:55:33Z

    Hi,

On 2023-03-15 08:14:09 +0100, Peter Eisentraut wrote:
> I have identified several open issues with the documentation build under
> Meson (approximately in priority order):
>
> 1. Image files are not handled at all, so they don't show up in the final
> product.

Hm. Somehow I thought I'd tackled that at some point. Ah. I got there for the
PDF output, but didn't realize it's also an issue for the html output.

For FO it sufficed to set the img.src.path param. For HTML that's not enough,
because that just adjusts the link to the file - but we don't want to link to
the source file. We actually solved this for the single-page html version - we
just embed the svg.  I wonder if we should just do that as well.

Another way would be to emit the files into the desired place as part of the
stylesheet. While it requires touching xslt, it does seems somewhat more
elegant than just copying files around. I did implement that, curious what you
think.


> 2. Defaults to website stylesheet, no way to configure.  This should be
> adjusted to match the make build.

Should we add a meson option?


> 3. The various build targets and their combinations are mismatching and
> incomplete.  For example:
>
> Top-level GNUmakefile has these targets:
>
> - docs (builds html and man)
> - html
> - man
>
> (Those are the formats that are part of a distribution build.)
>
> doc/src/sgml/Makefile has these documented targets:
>
> - default target is html
> - all (builds html and man, maps to top-level "docs")
> - html
> - man
> - postgres-A4.pdf
> - postgres-US.pdf
> - check
>
> as well as (undocumented):
>
> - htmlhelp
> - postgres.html
> - postgres.txt
> - epub
> - postgres.epub
> - postgres.info
>
> meson has the following documented targets:
>
> - docs (builds only html)
> - alldocs (builds all formats, including obscure ones)
>
> as well as the following undocumented targets:
>
> - html
> - man
> - html_help [sic]

renamed in the attached patch.


> - postgres-A4.pdf
> - postgres-US.pdf
> - postgres.epub

Note that these are actually named doc/src/sgml/{html,man,...}, not top-level
targets.


> - [info is not implemented at all]

Would be easy to implement, but not sure it's worth doing.


> - [didn't find an equivalent of check]

That's probably worth doing - should it be run as an actual test, or be a
target?


> 4. There doesn't appear to be a way to install the documentation.
> (There are also some open questions in the top-level meson.build about
> the installation directories, but I suppose if we can't install them
> then exactly where to install them hasn't been thought about too
> much.)

WIP patch for that attached. There's now
  install-doc-man
  install-doc-html
run targets and a
  install-docs
alias target.


I did end up getting stuck when hacking on this, and ended up adding css
support for nochunk and support for the website style for htmlhelp and
nochunk, as well as obsoleting the need for copying the css files... But
perhaps that's a bit too much.

Greetings,

Andres Freund

  3. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-03-20T02:33:38Z

    Hi,

On 2023-03-15 20:55:33 -0700, Andres Freund wrote:
> WIP patch for that attached. There's now
>   install-doc-man
>   install-doc-html
> run targets and a
>   install-docs
> alias target.
> 
> 
> I did end up getting stuck when hacking on this, and ended up adding css
> support for nochunk and support for the website style for htmlhelp and
> nochunk, as well as obsoleting the need for copying the css files... But
> perhaps that's a bit too much.

Updated set of patches attached. This one works in older meson versions too
and adds install-world and install-quiet targets.


I also ended up getting so frustrated at the docs build speed that I started
to hack a bit on that. I attached a patch shaving a few seconds off the
buildtime.


I think we can make the docs build in parallel and incrementally, by building
the different parts of the docs in parallel, using --stringparam rootid,
e.g. building each 'part' separately.

A very very rough draft attached:

parallel with parts:
real	0m10.831s
user	0m58.295s
sys	0m1.402s

normal:
real	0m32.215s
user	0m31.876s
sys	0m0.328s

1/3 of the build time at 2x the cost is nothing to sneeze at.

Greetings,

Andres Freund

  4. Re: meson documentation build open issues

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-03-20T10:58:08Z

    On 20.03.23 03:33, Andres Freund wrote:
>> I did end up getting stuck when hacking on this, and ended up adding css
>> support for nochunk and support for the website style for htmlhelp and
>> nochunk, as well as obsoleting the need for copying the css files... But
>> perhaps that's a bit too much.
> Updated set of patches attached. This one works in older meson versions too
> and adds install-world and install-quiet targets.

Oh, this patch set grew quite quickly. ;-)

[PATCH v2 1/8] meson: rename html_help target to htmlhelp

This is obvious.


[PATCH v2 5/8] docs: html: copy images to output as part of xslt build

Making the XSLT stylesheets do the copying has some appeal.  I think it 
would only work for SVG (or other XML) files, which I guess is okay, but 
maybe the templates should have a filter on format="SVG" or something. 
Also, this copying actually modifies the files in some XML-equivalent 
way.  Also okay, I think, but worth noting.

Note sure why you removed this comment

-<!-- strip directory name from image filerefs -->

since the code still exists.


[PATCH v2 6/8] wip: docs: copy or inline css

This seems pretty complicated compared to just copying a file?

  5. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-03-20T17:32:49Z

    Hi,

On 2023-03-20 11:58:08 +0100, Peter Eisentraut wrote:
> Oh, this patch set grew quite quickly. ;-)

Yep :)


> [PATCH v2 5/8] docs: html: copy images to output as part of xslt build
> 
> Making the XSLT stylesheets do the copying has some appeal.  I think it
> would only work for SVG (or other XML) files, which I guess is okay, but
> maybe the templates should have a filter on format="SVG" or something. Also,
> this copying actually modifies the files in some XML-equivalent way.  Also
> okay, I think, but worth noting.

I think it can be made work for non-xml files with xinclude too. But the
restriction around only working in top-level stylesheets (vs everywhere for
documents) is quite annoying.


> [PATCH v2 6/8] wip: docs: copy or inline css
> 
> This seems pretty complicated compared to just copying a file?

Mainly that it works correctly for the standalone file.

Greetings,

Andres Freund

  6. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-03-20T19:16:39Z

    Hi,

On 2023-03-19 19:33:38 -0700, Andres Freund wrote:
> I think we can make the docs build in parallel and incrementally, by building
> the different parts of the docs in parallel, using --stringparam rootid,
> e.g. building each 'part' separately.
> 
> A very very rough draft attached:
> 
> parallel with parts:
> real	0m10.831s
> user	0m58.295s
> sys	0m1.402s
> 
> normal:
> real	0m32.215s
> user	0m31.876s
> sys	0m0.328s
> 
> 1/3 of the build time at 2x the cost is nothing to sneeze at.

I could not make myself stop trying to figure out where the big constant time
factor comes from. Every invocation costs about 2s, even if not much is
rendered. Turns out, that's solely spent building all the <xsl:key>s. The
first time *any* key() is invoked for a document, all the keys are computed in
a single pass over the document.

A single reasonable key doesn't take that much time, even for the size of our
docs. But there are several redundant keys being built. Some of them somewhat
expensive. E.g. each
<xsl:key name="genid" match="*" use="generate-id()"/>
takes about 300ms. There's one in chunk-common and one in
docbook-no-doctype.xsl.

I'm going to cry now.

Greetings,

Andres Freund

  7. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-03-22T18:59:17Z

    Hi,

On 2023-03-20 10:32:49 -0700, Andres Freund wrote:
> On 2023-03-20 11:58:08 +0100, Peter Eisentraut wrote:
> > Oh, this patch set grew quite quickly. ;-)
> 
> Yep :)

Unless somebody sees a reason to wait, I am planning to commit:
  meson: add install-{quiet, world} targets
  meson: add install-{docs,doc-html,doc-man} targets
  meson: make install_test_files more generic, rename to install_files

While I don't think we have necessarily the path forward around .css and .svg,
the above are independent of that.


For the .svg: I wonder if we should just inline the images in the chunked
html, just like we do in the single page one. It's not like we reuse one image
across a lot of pages, so there's no bandwidth saved from having the images
separate...

For the .css: docbook-xsl actually has support for writing the .css: [1] - but
it requires the .css file be valid xml. I wonder if the cleanest approch would
be to have a build step to create .css.xml - then the non-chunked build's
generate.css.header would do the right thing.


I'll start a new thread for
  docs: speed up docs build by special-casing the gentext.template
  VERY WIP: parallel doc generation
after the feature freeze.


After looking into it a tiny bit more, it seems we should use neither pandoc
nor dbtoepub for epub generation.

All the dbtoepub does is to invoke the docbook-xsl support for epubs and zip
the result - except it doesn't use our stylesheets, so it looks randomly
different and doesn't use our speedups. At the very least we should use our
customizations, if we want epub support. Or we should just remove it.

Pandoc unfortunately doesn't do docbook well enough to be usable for now to
directly parse our docbook.

Regards,

Andres

[1] https://docbook.sourceforge.net/release/xsl/current/doc/html/custom.css.source.html

  8. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-03-24T07:26:19Z

    Hi,

On 2023-03-22 11:59:17 -0700, Andres Freund wrote:
> Unless somebody sees a reason to wait, I am planning to commit:
>   meson: add install-{quiet, world} targets
>   meson: add install-{docs,doc-html,doc-man} targets
>   meson: make install_test_files more generic, rename to install_files

I've done that now.


> For the .css: docbook-xsl actually has support for writing the .css: [1] - but
> it requires the .css file be valid xml. I wonder if the cleanest approch would
> be to have a build step to create .css.xml - then the non-chunked build's
> generate.css.header would do the right thing.

We don't even need to do that! The attached patch just creates a wrapper
css.xml that loads the .css via an entity reference.

I think this looks reasonably complicated, given that it gives us a working
stylesheet for the non-chunked output?

I don't know if my hack of putting the paramters in stylesheet-common.xsl is
reasonable. Perhaps we should just include stylesheet-html-common.xsl in
stylesheet-hh.xsl, then this uglyness wouldn't be required.


Greetings,

Andres Freund

  9. Re: meson documentation build open issues

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-03-24T10:59:23Z

    On 24.03.23 08:26, Andres Freund wrote:
>> For the .css: docbook-xsl actually has support for writing the .css: [1] - but
>> it requires the .css file be valid xml. I wonder if the cleanest approch would
>> be to have a build step to create .css.xml - then the non-chunked build's
>> generate.css.header would do the right thing.
> 
> We don't even need to do that! The attached patch just creates a wrapper
> css.xml that loads the .css via an entity reference.

That looks like a better solution.

> I don't know if my hack of putting the paramters in stylesheet-common.xsl is
> reasonable. Perhaps we should just include stylesheet-html-common.xsl in
> stylesheet-hh.xsl, then this uglyness wouldn't be required.

Maybe, but it's not clear whether all the customizations in there are 
applicable to htmlhelp.

Another option here is to remove support for htmlhelp.

  10. Remove 'htmlhelp' documentat format (was meson documentation build open issues)

    Andres Freund <andres@anarazel.de> — 2023-03-24T16:58:22Z

    Hi,

On 2023-03-24 11:59:23 +0100, Peter Eisentraut wrote:
> Another option here is to remove support for htmlhelp.

That might actually be the best path - it certainly doesn't look like anybody
has been actively using it. Or otherwise somebody would have complained about
there not being any instructions on how to actually compile a .chm file. And
perhaps complained that it takes next to forever to build.

I also have the impression that people don't use the .chm stuff much anymore,
but that might just be me not using windows.

Greetings,

Andres

  11. Re: Remove 'htmlhelp' documentat format (was meson documentation build open issues)

    Daniel Gustafsson <daniel@yesql.se> — 2023-03-24T21:00:56Z

    > On 24 Mar 2023, at 17:58, Andres Freund <andres@anarazel.de> wrote:
> On 2023-03-24 11:59:23 +0100, Peter Eisentraut wrote:
>> Another option here is to remove support for htmlhelp.
> 
> That might actually be the best path - it certainly doesn't look like anybody
> has been actively using it.

I had no idea we had support for building a .chm until reading this, but I've
also never come across anyone asking for such a docset.  FWIW, no objections to
it going.

--
Daniel Gustafsson

  12. Re: Remove 'htmlhelp' documentat format (was meson documentation build open issues)

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-03-28T09:46:41Z

    On 24.03.23 17:58, Andres Freund wrote:
> On 2023-03-24 11:59:23 +0100, Peter Eisentraut wrote:
>> Another option here is to remove support for htmlhelp.
> 
> That might actually be the best path - it certainly doesn't look like anybody
> has been actively using it. Or otherwise somebody would have complained about
> there not being any instructions on how to actually compile a .chm file. And
> perhaps complained that it takes next to forever to build.
> 
> I also have the impression that people don't use the .chm stuff much anymore,
> but that might just be me not using windows.

I think in ancient times, pgadmin used it for its internal help.

But I have heard less about htmlhelp over the years than about the info 
format.

  13. Re: Remove 'htmlhelp' documentat format (was meson documentation build open issues)

    Dave Page <dpage@pgadmin.org> — 2023-03-28T09:50:07Z

    On Tue, 28 Mar 2023 at 10:46, Peter Eisentraut <
peter.eisentraut@enterprisedb.com> wrote:

> On 24.03.23 17:58, Andres Freund wrote:
> > On 2023-03-24 11:59:23 +0100, Peter Eisentraut wrote:
> >> Another option here is to remove support for htmlhelp.
> >
> > That might actually be the best path - it certainly doesn't look like
> anybody
> > has been actively using it. Or otherwise somebody would have complained
> about
> > there not being any instructions on how to actually compile a .chm file.
> And
> > perhaps complained that it takes next to forever to build.
> >
> > I also have the impression that people don't use the .chm stuff much
> anymore,
> > but that might just be me not using windows.
>
> I think in ancient times, pgadmin used it for its internal help.
>

Yes, very ancient :-). We use Sphinx now.


>
> But I have heard less about htmlhelp over the years than about the info
> format.
>
>
>

-- 
Dave Page
Blog: https://pgsnake.blogspot.com
Twitter: @pgsnake

EDB: https://www.enterprisedb.com

  14. Re: meson documentation build open issues

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-04-05T10:24:04Z

    On 15.03.23 08:14, Peter Eisentraut wrote:
> I have identified several open issues with the documentation build under 
> Meson (approximately in priority order):

Some work has been done on this.  Here is my current assessment.

> 1. Image files are not handled at all, so they don't show up in the 
> final product.

This is fixed.

> 2. Defaults to website stylesheet, no way to configure.  This should be 
> adjusted to match the make build.

This is fixed.

> 3. The various build targets and their combinations are mismatching and 
> incomplete.

This has been improved, and I see there is documentation.

I think it's still an issue that "make docs" builds html and man but 
"ninja docs" only builds html.  For some reason the wiki page actually 
claims that ninja docs builds both, but this does not happen for me.

> 4. There doesn't appear to be a way to install the documentation.

This has been addressed.

> 5. There doesn't appear to be an equivalent of "make world" and "make
> install-world" that includes documentation builds.

This has been addressed with the additional meson auto options.  But it 
seems that these options only control building, not installing, so there 
is no "install-world" aspect yet.

  15. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-04-05T14:45:12Z

    Hi,

On 2023-04-05 12:24:04 +0200, Peter Eisentraut wrote:
> On 15.03.23 08:14, Peter Eisentraut wrote:
> > 3. The various build targets and their combinations are mismatching and
> > incomplete.
> 
> This has been improved, and I see there is documentation.
> 
> I think it's still an issue that "make docs" builds html and man but "ninja
> docs" only builds html.  For some reason the wiki page actually claims that
> ninja docs builds both, but this does not happen for me.

It used to, but Tom insisted that it should not. I'm afraid that it's not
quite possible to emulate make here. 'make docs' at the toplevel builds both
HTML and manpages. But 'make -C doc/src/sgml', only builds HTML.


> > 5. There doesn't appear to be an equivalent of "make world" and "make
> > install-world" that includes documentation builds.
> 
> This has been addressed with the additional meson auto options.  But it
> seems that these options only control building, not installing, so there is
> no "install-world" aspect yet.

I'm not following - install-world install docs if the docs feature is
available, and not if not?

Greetings,

Andres Freund

  16. Re: meson documentation build open issues

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-04-06T09:11:57Z

    On 05.04.23 16:45, Andres Freund wrote:
>> I think it's still an issue that "make docs" builds html and man but "ninja
>> docs" only builds html.  For some reason the wiki page actually claims that
>> ninja docs builds both, but this does not happen for me.
> 
> It used to, but Tom insisted that it should not. I'm afraid that it's not
> quite possible to emulate make here. 'make docs' at the toplevel builds both
> HTML and manpages. But 'make -C doc/src/sgml', only builds HTML.

Ok, not a topic for this thread then.

>>> 5. There doesn't appear to be an equivalent of "make world" and "make
>>> install-world" that includes documentation builds.
>>
>> This has been addressed with the additional meson auto options.  But it
>> seems that these options only control building, not installing, so there is
>> no "install-world" aspect yet.
> 
> I'm not following - install-world install docs if the docs feature is
> available, and not if not?

I had expected that if meson setup enables the 'docs' feature, then 
meson compile will build the documentation, which happens, and meson 
install will install it, which does not happen.

  17. Re: meson documentation build open issues

    Andrew Dunstan <andrew@dunslane.net> — 2023-04-07T14:39:56Z

    On 2023-04-06 Th 05:11, Peter Eisentraut wrote:
> On 05.04.23 16:45, Andres Freund wrote:
>>> I think it's still an issue that "make docs" builds html and man but 
>>> "ninja
>>> docs" only builds html.  For some reason the wiki page actually 
>>> claims that
>>> ninja docs builds both, but this does not happen for me.
>>
>> It used to, but Tom insisted that it should not. I'm afraid that it's 
>> not
>> quite possible to emulate make here. 'make docs' at the toplevel 
>> builds both
>> HTML and manpages. But 'make -C doc/src/sgml', only builds HTML.
>
> Ok, not a topic for this thread then.
>
>>>> 5. There doesn't appear to be an equivalent of "make world" and "make
>>>> install-world" that includes documentation builds.
>>>
>>> This has been addressed with the additional meson auto options.  But it
>>> seems that these options only control building, not installing, so 
>>> there is
>>> no "install-world" aspect yet.
>>
>> I'm not following - install-world install docs if the docs feature is
>> available, and not if not?
>
> I had expected that if meson setup enables the 'docs' feature, then 
> meson compile will build the documentation, which happens, and meson 
> install will install it, which does not happen.
>
>
>

"meson compile" doesn't seem to build the docs by default ( see 
<https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=crake&dt=2023-04-06%2018%3A17%3A04&stg=build>), 
and I'd rather it didn't, building the docs is a separate and optional 
step for the buildfarm.


cheers


andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

  18. Re: meson documentation build open issues

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2023-04-12T15:30:28Z

    On 07.04.23 16:39, Andrew Dunstan wrote:
>>>>> 5. There doesn't appear to be an equivalent of "make world" and "make
>>>>> install-world" that includes documentation builds.
>>>>
>>>> This has been addressed with the additional meson auto options.  But it
>>>> seems that these options only control building, not installing, so 
>>>> there is
>>>> no "install-world" aspect yet.
>>>
>>> I'm not following - install-world install docs if the docs feature is
>>> available, and not if not?
>>
>> I had expected that if meson setup enables the 'docs' feature, then 
>> meson compile will build the documentation, which happens, and meson 
>> install will install it, which does not happen.
> 
> "meson compile" doesn't seem to build the docs by default ( see 
> <https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=crake&dt=2023-04-06%2018%3A17%3A04&stg=build>), and I'd rather it didn't, building the docs is a separate and optional step for the buildfarm.

You can control this with the "docs" option for meson, as of recently.

  19. Re: meson documentation build open issues

    Christoph Berg <myon@debian.org> — 2023-11-03T14:26:05Z

    Re: Peter Eisentraut
> > "meson compile" doesn't seem to build the docs by default ( see <https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=crake&dt=2023-04-06%2018%3A17%3A04&stg=build>),
> > and I'd rather it didn't, building the docs is a separate and optional
> > step for the buildfarm.
> 
> You can control this with the "docs" option for meson, as of recently.

I've been looking into switching the Debian PG 17 build to meson, but
I'm running into several problems.

* The docs are still not built by default, and -Ddocs=enabled doesn't
  change that

* None of the "build docs" targets are documented in install-meson.html

* "ninja -C build alldocs" works, but it's impossible to see what
  flavors it's actually building. Everything is autodetected, and
  perhaps I would like to no build the .txt/something variants,
  but I have no idea what switch that is, or what package I have to
  uninstall so it's not autodetected (only html and pdf are
  documented.)

  Are there any other targets for the individual formats? (I could
  probably use one for the manpages only, without the html.)

Non-doc issues:

* LLVM is off by default (ok), when I enable it with -Dllvm=auto, it
  gets detected, but no .bc files are built, nor installed

* selinux is not autodetected. It needs -Dselinux=auto, but that's not
  documented in install-meson.html

* There is no split between libdir and pkglibdir. We had used that in
  the past for libpq -> /usr/lib/x86_64-linux-gnu and PG stuff ->
  /usr/lib/postgresql/17/lib.

Christoph

  20. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-11-03T16:38:48Z

    Hi,

On 2023-11-03 15:26:05 +0100, Christoph Berg wrote:
> Re: Peter Eisentraut
> > > "meson compile" doesn't seem to build the docs by default ( see <https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=crake&dt=2023-04-06%2018%3A17%3A04&stg=build>),
> > > and I'd rather it didn't, building the docs is a separate and optional
> > > step for the buildfarm.
> >
> > You can control this with the "docs" option for meson, as of recently.
>
> I've been looking into switching the Debian PG 17 build to meson, but
> I'm running into several problems.
>
> * The docs are still not built by default, and -Ddocs=enabled doesn't
>   change that

Maybe I am missing something - they aren't built by default in autoconf
either?


> * None of the "build docs" targets are documented in install-meson.html

Hm, odd, I thought they were, but you are right. There were some docs patches
that we never really could find agreement upon :/


> * "ninja -C build alldocs" works, but it's impossible to see what
>   flavors it's actually building. Everything is autodetected, and
>   perhaps I would like to no build the .txt/something variants,
>   but I have no idea what switch that is, or what package I have to
>   uninstall so it's not autodetected (only html and pdf are
>   documented.)

I think a package build should probably turn off auto-detection (
  meson setup --auto-features=disabled) and enable specific features that are
desired - in which case you get errors if they are not available. Which
presumably is the behaviour you'd like?



>   Are there any other targets for the individual formats? (I could
>   probably use one for the manpages only, without the html.)

Yes, there are.
ninja doc/src/sgml/{postgres-A4.pdf,html,postgres.html,man1}

Perhaps more interesting for your purposes, there are the
install-doc-{html,man} targets.

I remember discussing adding doc-{html,man} targets alongside
install-doc-{html,man}, not sure why we ended up not doing that. I'd be in
favor of adding them.

I've also been wondering about a 'help' target that documents important
targets in a interactively usable way.


> Non-doc issues:
>
> * LLVM is off by default (ok), when I enable it with -Dllvm=auto, it
>   gets detected, but no .bc files are built, nor installed

Support for that has not yet been merged.


> * selinux is not autodetected. It needs -Dselinux=auto, but that's not
>   documented in install-meson.html

Uh, huh. There's no documentation for --with-selinux in the installation.sgml
either, just in sepgsql.sgml. So when the relevant docs got translated to
meson, -Dselinux= wasn't documented either.


> * There is no split between libdir and pkglibdir. We had used that in
>   the past for libpq -> /usr/lib/x86_64-linux-gnu and PG stuff ->
>   /usr/lib/postgresql/17/lib.

I don't think the autoconf build currently exposes separately configuring
pkglibdir either, I think that's a debian patch? I'm entirely open to adding
an explicit configuration option for this though.


Thanks for looking at this, it's quite helpful!

Greetings,

Andres Freund

  21. Re: meson documentation build open issues

    Christoph Berg <myon@debian.org> — 2023-11-03T18:19:17Z

    Re: Andres Freund
> > > You can control this with the "docs" option for meson, as of recently.
> >
> > I've been looking into switching the Debian PG 17 build to meson, but
> > I'm running into several problems.
> >
> > * The docs are still not built by default, and -Ddocs=enabled doesn't
> >   change that
> 
> Maybe I am missing something - they aren't built by default in autoconf
> either?

True, but the documentation (and this thread) reads like it should. Or
at least it should, when I explicitly say -Ddocs=enabled.

What would also help is when the tail of the meson output had a list
of features that are enabled. There's the list of "External libraries"
which is quite helpful at figuring out what's still missing, but
perhaps this could be extended:

  Features
    LLVM     : YES (/usr/bin/llvm-config-16)
    DOCS     : YES (html pdf texinfo)

Atm it's hidden in the long initial blurb of "Checking for.." and the
"NO" in there don't really stand out as much, since some of them are
normal.

> > * "ninja -C build alldocs" works, but it's impossible to see what
> >   flavors it's actually building. Everything is autodetected, and
> >   perhaps I would like to no build the .txt/something variants,
> >   but I have no idea what switch that is, or what package I have to
> >   uninstall so it's not autodetected (only html and pdf are
> >   documented.)
> 
> I think a package build should probably turn off auto-detection (
>   meson setup --auto-features=disabled) and enable specific features that are
> desired - in which case you get errors if they are not available. Which
> presumably is the behaviour you'd like?

I'm still trying to figure out the best spot in that space of options.
Currently I'm still in the phase of getting it to work at all; the end
result might well use that option.

> >   Are there any other targets for the individual formats? (I could
> >   probably use one for the manpages only, without the html.)
> 
> Yes, there are.
> ninja doc/src/sgml/{postgres-A4.pdf,html,postgres.html,man1}

Oh, that was not obvious to me that this "make $some_file" style
command would work. (But it still leaves the problem of knowing which
targets there are.)

> Perhaps more interesting for your purposes, there are the
> install-doc-{html,man} targets.

Hmm, I thought I had tried these, but apparently managed to miss them.
Thanks.

install-doc-man seems to install "man1" only, though?
(It seems to compile man5/man7, but not install them.)

> I remember discussing adding doc-{html,man} targets alongside
> install-doc-{html,man}, not sure why we ended up not doing that. I'd be in
> favor of adding them.
> 
> I've also been wondering about a 'help' target that documents important
> targets in a interactively usable way.

That is definitely missing, yes. I found out about "alldocs" only
after reading the meson files, and that took more than it should have.

> > Non-doc issues:
> >
> > * LLVM is off by default (ok), when I enable it with -Dllvm=auto, it
> >   gets detected, but no .bc files are built, nor installed
> 
> Support for that has not yet been merged.

Oh, that's a showstopper. I thought meson would already be ready for
production use. There is indeed an "experimental" note in
install-requirements.html, but not in install-meson.html

> > * selinux is not autodetected. It needs -Dselinux=auto, but that's not
> >   documented in install-meson.html
> 
> Uh, huh. There's no documentation for --with-selinux in the installation.sgml
> either, just in sepgsql.sgml. So when the relevant docs got translated to
> meson, -Dselinux= wasn't documented either.

Ok. It does show up in "External libraries" and was enabled in the
Debian packages before.

Why isn't it "auto" like the others?

> > * There is no split between libdir and pkglibdir. We had used that in
> >   the past for libpq -> /usr/lib/x86_64-linux-gnu and PG stuff ->
> >   /usr/lib/postgresql/17/lib.
> 
> I don't think the autoconf build currently exposes separately configuring
> pkglibdir either, I think that's a debian patch? I'm entirely open to adding
> an explicit configuration option for this though.

That would definitely be helpful.

> Thanks for looking at this, it's quite helpful!

Thanks for the feedback!
Christoph

  22. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-11-03T18:53:44Z

    Hi,

On 2023-11-03 19:19:17 +0100, Christoph Berg wrote:
> Re: Andres Freund
> > > > You can control this with the "docs" option for meson, as of recently.
> > >
> > > I've been looking into switching the Debian PG 17 build to meson, but
> > > I'm running into several problems.
> > >
> > > * The docs are still not built by default, and -Ddocs=enabled doesn't
> > >   change that
> >
> > Maybe I am missing something - they aren't built by default in autoconf
> > either?
>
> True, but the documentation (and this thread) reads like it should. Or
> at least it should, when I explicitly say -Ddocs=enabled.

My understanding of the intent of the options is to make meson error out if
the required dependencies are not available, not that it controls when the
build targets are built.

The reason for that is simply that the docs take too long to build.


> What would also help is when the tail of the meson output had a list
> of features that are enabled. There's the list of "External libraries"
> which is quite helpful at figuring out what's still missing, but
> perhaps this could be extended:
>
>   Features
>     LLVM     : YES (/usr/bin/llvm-config-16)
>     DOCS     : YES (html pdf texinfo)
>
> Atm it's hidden in the long initial blurb of "Checking for.." and the
> "NO" in there don't really stand out as much, since some of them are
> normal.

The summary does include both. LLVM is 'llvm', man/html docs is 'docs' and pdf
docs as 'docs_pdf'.


> > >   Are there any other targets for the individual formats? (I could
> > >   probably use one for the manpages only, without the html.)
> >
> > Yes, there are.
> > ninja doc/src/sgml/{postgres-A4.pdf,html,postgres.html,man1}
>
> Oh, that was not obvious to me that this "make $some_file" style
> command would work. (But it still leaves the problem of knowing which
> targets there are.)

Yes, you can trigger building any file that way.

The following is *not* an argument the docs targets shouldn't be documented
(working on a patch), just something that might be helpful until then /
separately. You can see which targets are built with

ninja -t targets all|grep doc/src/


> > Perhaps more interesting for your purposes, there are the
> > install-doc-{html,man} targets.
>
> Hmm, I thought I had tried these, but apparently managed to miss them.
> Thanks.
>
> install-doc-man seems to install "man1" only, though?
> (It seems to compile man5/man7, but not install them.)

Ugh, that's obviously a bug. I'll fix it.


> > > Non-doc issues:
> > >
> > > * LLVM is off by default (ok), when I enable it with -Dllvm=auto, it
> > >   gets detected, but no .bc files are built, nor installed
> >
> > Support for that has not yet been merged.
>
> Oh, that's a showstopper. I thought meson would already be ready for
> production use. There is indeed an "experimental" note in
> install-requirements.html, but not in install-meson.html

I'm working on merging it. Having it for core PG isn't a huge difficulty, the
extension story is what's been holding me back...


> > > * selinux is not autodetected. It needs -Dselinux=auto, but that's not
> > >   documented in install-meson.html
> >
> > Uh, huh. There's no documentation for --with-selinux in the installation.sgml
> > either, just in sepgsql.sgml. So when the relevant docs got translated to
> > meson, -Dselinux= wasn't documented either.
>
> Ok. It does show up in "External libraries" and was enabled in the
> Debian packages before.
>
> Why isn't it "auto" like the others?

I don't really remember why I did that, but it's platform specific, maybe
that's why I did it that way?


> > > * There is no split between libdir and pkglibdir. We had used that in
> > >   the past for libpq -> /usr/lib/x86_64-linux-gnu and PG stuff ->
> > >   /usr/lib/postgresql/17/lib.
> >
> > I don't think the autoconf build currently exposes separately configuring
> > pkglibdir either, I think that's a debian patch? I'm entirely open to adding
> > an explicit configuration option for this though.
>
> That would definitely be helpful.

I have a patch locally, will send it together with a few others in a bit.

Greetings,

Andres Freund

  23. Re: meson documentation build open issues

    Christoph Berg <myon@debian.org> — 2023-11-03T19:19:18Z

    Re: Andres Freund
> The reason for that is simply that the docs take too long to build.

That why I'd prefer to be able to separate arch:all and arch:any
builds, yes.

> The summary does include both. LLVM is 'llvm', man/html docs is 'docs' and pdf
> docs as 'docs_pdf'.

Sorry, I should have looked closer. :(

> The following is *not* an argument the docs targets shouldn't be documented
> (working on a patch), just something that might be helpful until then /
> separately. You can see which targets are built with
> 
> ninja -t targets all|grep doc/src/

Thanks.

> > Oh, that's a showstopper. I thought meson would already be ready for
> > production use. There is indeed an "experimental" note in
> > install-requirements.html, but not in install-meson.html
> 
> I'm working on merging it. Having it for core PG isn't a huge difficulty, the
> extension story is what's been holding me back...

In-core extensions or external ones?

> > Why isn't it "auto" like the others?
> 
> I don't really remember why I did that, but it's platform specific, maybe
> that's why I did it that way?

Isn't that kind the point of autodetecting things? Aren't bonjour and
bsd_auth autodetected as well?

> > > I don't think the autoconf build currently exposes separately configuring
> > > pkglibdir either, I think that's a debian patch? I'm entirely open to adding
> > > an explicit configuration option for this though.
> >
> > That would definitely be helpful.
> 
> I have a patch locally, will send it together with a few others in a bit.

Thanks!

Christoph

  24. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-11-03T21:16:01Z

    Hi,


On 2023-11-03 20:19:18 +0100, Christoph Berg wrote:
> Re: Andres Freund
> > The reason for that is simply that the docs take too long to build.
>
> That why I'd prefer to be able to separate arch:all and arch:any
> builds, yes.

What's stopping you from doing that?  I think the only arch:any content we
have is the docs, and those you can build separately? Doc builds do trigger
generation of a handful of files besides the docs, but not more.


> > > Oh, that's a showstopper. I thought meson would already be ready for
> > > production use. There is indeed an "experimental" note in
> > > install-requirements.html, but not in install-meson.html
> >
> > I'm working on merging it. Having it for core PG isn't a huge difficulty, the
> > extension story is what's been holding me back...
>
> In-core extensions or external ones?

Both, although the difficulty of doing it is somewhat separate for each.


> > > Why isn't it "auto" like the others?
> >
> > I don't really remember why I did that, but it's platform specific, maybe
> > that's why I did it that way?
>
> Isn't that kind the point of autodetecting things? Aren't bonjour and
> bsd_auth autodetected as well?

I'd be happy to change it, unless somebody objects?


> > > > I don't think the autoconf build currently exposes separately configuring
> > > > pkglibdir either, I think that's a debian patch? I'm entirely open to adding
> > > > an explicit configuration option for this though.
> > >
> > > That would definitely be helpful.
> >
> > I have a patch locally, will send it together with a few others in a bit.
>
> Thanks!

Attached.

0001 - the bugfix for install-man only installing man1, I'll push that soon
0002 - Document --with-selinux/-Dselinux options centrally
0003 - Add doc-{html,man} targets

       I'm not quite sure it's worth it, but it's basically free, so ...

0004 - Documentation for important build targets

       I'm not entirely happy with the formatting, but it looks like that's
       mostly a CSS issue. I started a thread on fixing that on -www.

0005 - Add -Dpkglibdir option

       I guess we might want to do the same for configure if we decide to do
       this?

Greetings,

Andres Freund

  25. Re: meson documentation build open issues

    Christoph Berg <myon@debian.org> — 2023-11-06T09:45:27Z

    Re: Andres Freund
> > > The reason for that is simply that the docs take too long to build.
> >
> > That why I'd prefer to be able to separate arch:all and arch:any
> > builds, yes.
> 
> What's stopping you from doing that?  I think the only arch:any content we
> have is the docs, and those you can build separately? Doc builds do trigger
> generation of a handful of files besides the docs, but not more.

Historically, .deb files have been required to contain the manpages
for all executables even when there's a separate -doc package. This
means we'd need a separate (hopefully fast) manpage build even when
the arch:any binaries are built. We might get around that by
introducing a new postgresql-manpages-XX arch:all package, but that
might be too much micropackaging.

The install-doc-man target will likely fix it, will play with it a bit
more, thanks.

> > > I'm working on merging it. Having it for core PG isn't a huge difficulty, the
> > > extension story is what's been holding me back...
> >
> > In-core extensions or external ones?
> 
> Both, although the difficulty of doing it is somewhat separate for each.

I'd think most external extensions could stay with pgxs.mk for the
time being.


> +     <varlistentry id="configure-with-sepgsql-meson">
> +      <term><option>-Dselinux={ disabled | auto | enabled }</option></term>
> +      <listitem>
> +       <para>
> +        Build with selinux support, enabling the <xref linkend="sepgsql"/>
> +        extension.

This option defaults to ... auto?


> index 90e2c062fa8..003b57498bb 100644
> --- a/doc/src/sgml/meson.build
> +++ b/doc/src/sgml/meson.build
> @@ -142,6 +142,7 @@ if docs_dep.found()
>        '--install-dir-contents', dir_doc_html, html],
>      build_always_stale: true, build_by_default: false,
>    )
> +  alias_target('doc-html', install_doc_html)
>    alias_target('install-doc-html', install_doc_html)

Shouldn't this just build the html docs, without installing?

> +  alias_target('doc-man', install_doc_html)
>    alias_target('install-doc-man', install_doc_man)

... same


> +     <varlistentry id="meson-target-install-world">
> +       <term><option>install-install-world</option></term>

install-world

> +     <varlistentry id="meson-target-install-doc-man">
> +       <term><option>install-doc-html</option></term>
> +       <listitem>
> +        <para>
> +         Install documentation in man page format.

install-doc-man

> +   <sect3 id="meson-targets-docs">
> +    <title>Documentation Targets</title>

> +     <varlistentry id="meson-target-docs">
> +       <term><option>docs</option></term>
> +       <term><option>doc-html</option></term>
> +       <listitem>
> +        <para>
> +         Build documentation in multi-page HTML format.  Note that
> +         <option>docs</option> does <emphasis>not</emphasis> include building
> +         man page documentation, as man page generation seldom fails when
> +         building HTML documentation succeeds.

Why is that a reason for not building the manpages?

> +   <sect3 id="meson-targets-code">
> +    <title>Code Targets</title>

I would have expected the sections to be in the order
build-docs-install. Having install first seems weird to me.

> +   <sect3 id="meson-targets-other">
> +    <title>Other Targets</title>
> +
> +    <variablelist>
> +
> +     <varlistentry id="meson-target-clean">
> +       <term><option>clean</option></term>
> +       <listitem>
> +        <para>
> +         Remove all build products
> +        </para>
> +       </listitem>
> +     </varlistentry>
> +
> +     <varlistentry id="meson-target-test">
> +       <term><option>test</option></term>
> +       <listitem>
> +        <para>
> +         Remove all enabled tests. Support for some classes of tests can be
> +         enabled / disabled with <xref linkend="configure-tap-tests-meson"/>
> +         and <xref linkend="configure-pg-test-extra-meson"/>.

This should explicitly say if contrib tests are included (or there
needs to be a separate test-world target.)


> Subject: [PATCH v1 5/5] meson: Add -Dpkglibdir option

Will give that a try, thanks!

Christoph

  26. Re: meson documentation build open issues

    Peter Eisentraut <peter@eisentraut.org> — 2023-11-07T15:44:42Z

    On 03.11.23 22:16, Andres Freund wrote:
[selinux option]
>>>> Why isn't it "auto" like the others?
>>> I don't really remember why I did that, but it's platform specific, maybe
>>> that's why I did it that way?
>> Isn't that kind the point of autodetecting things? Aren't bonjour and
>> bsd_auth autodetected as well?
> I'd be happy to change it, unless somebody objects?

Makes sense to me to change it to auto.

  27. Re: meson documentation build open issues

    Peter Eisentraut <peter@eisentraut.org> — 2023-11-07T15:55:37Z

    On 03.11.23 19:19, Christoph Berg wrote:
>>>> You can control this with the "docs" option for meson, as of recently.
>>> I've been looking into switching the Debian PG 17 build to meson, but
>>> I'm running into several problems.
>>>
>>> * The docs are still not built by default, and -Ddocs=enabled doesn't
>>>    change that
>> Maybe I am missing something - they aren't built by default in autoconf
>> either?
> True, but the documentation (and this thread) reads like it should. Or
> at least it should, when I explicitly say -Ddocs=enabled.
> 
> What would also help is when the tail of the meson output had a list
> of features that are enabled. There's the list of "External libraries"
> which is quite helpful at figuring out what's still missing, but
> perhaps this could be extended:
> 
>    Features
>      LLVM     : YES (/usr/bin/llvm-config-16)
>      DOCS     : YES (html pdf texinfo)
> 
> Atm it's hidden in the long initial blurb of "Checking for.." and the
> "NO" in there don't really stand out as much, since some of them are
> normal.

I don't feel like we have fully worked out how the docs options should 
fit together.

With the make build system, there is a canonical sequence of

make world
make check-world
make install-world

that encompasses everything.

Now with meson to handle the documentation one needs to remember a 
variety of additional targets.  (There is a risk that once this gets 
more widespread, more people will submit broken documentation.)

I would like to have some set of options that enables it so that the 
standard documentation targets become part of "meson compile" and "meson 
install".

  28. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-11-07T16:08:29Z

    Hi, 

On November 7, 2023 7:55:37 AM PST, Peter Eisentraut <peter@eisentraut.org> wrote:
>On 03.11.23 19:19, Christoph Berg wrote:
>>>>> You can control this with the "docs" option for meson, as of recently.
>>>> I've been looking into switching the Debian PG 17 build to meson, but
>>>> I'm running into several problems.
>>>> 
>>>> * The docs are still not built by default, and -Ddocs=enabled doesn't
>>>>    change that
>>> Maybe I am missing something - they aren't built by default in autoconf
>>> either?
>> True, but the documentation (and this thread) reads like it should. Or
>> at least it should, when I explicitly say -Ddocs=enabled.
>> 
>> What would also help is when the tail of the meson output had a list
>> of features that are enabled. There's the list of "External libraries"
>> which is quite helpful at figuring out what's still missing, but
>> perhaps this could be extended:
>> 
>>    Features
>>      LLVM     : YES (/usr/bin/llvm-config-16)
>>      DOCS     : YES (html pdf texinfo)
>> 
>> Atm it's hidden in the long initial blurb of "Checking for.." and the
>> "NO" in there don't really stand out as much, since some of them are
>> normal.
>
>I don't feel like we have fully worked out how the docs options should fit together.
>
>With the make build system, there is a canonical sequence of
>
>make world
>make check-world
>make install-world
>
>that encompasses everything.
>
>Now with meson to handle the documentation one needs to remember a variety of additional targets.  (There is a risk that once this gets more widespread, more people will submit broken documentation.)

install-world with meson also installs docs.


>I would like to have some set of options that enables it so that the standard documentation targets become part of "meson compile" and "meson install".

-0.5 - it's just too painfully slow. For all scripted uses you can just as well use install-world...

Andres
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

  29. Re: meson documentation build open issues

    Tom Lane <tgl@sss.pgh.pa.us> — 2023-11-07T16:34:08Z

    Andres Freund <andres@anarazel.de> writes:
> On November 7, 2023 7:55:37 AM PST, Peter Eisentraut <peter@eisentraut.org> wrote:
>> I would like to have some set of options that enables it so that the standard documentation targets become part of "meson compile" and "meson install".

> -0.5 - it's just too painfully slow. For all scripted uses you can just as well use install-world...

I think we should set up the meson stuff so that "install" and
"install-world" cover exactly what they did with "make".  Otherwise
there will be too much confusion.

			regards, tom lane

  30. Re: meson documentation build open issues

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2023-11-07T16:40:40Z

    On 2023-Nov-07, Andres Freund wrote:

> >I would like to have some set of options that enables it so that the
> >standard documentation targets become part of "meson compile" and
> >"meson install".
> 
> -0.5 - it's just too painfully slow. For all scripted uses you can just as well use install-world...

If the problem is broken doc patches, then maybe a solution is to
include the `xmllint --noout --valid` target in whatever the check-world
equivalent is for meson.  Looking at doc/src/sgml/meson.build, we don't
seem to do that anywhere.  Doing the no-output lint run is very fast
(375ms real time in my machine, whereas "make html" takes 27s).

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"¿Cómo puedes confiar en algo que pagas y que no ves,
y no confiar en algo que te dan y te lo muestran?" (Germán Poo)

  31. Re: meson documentation build open issues

    Tom Lane <tgl@sss.pgh.pa.us> — 2023-11-07T16:49:05Z

    Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> If the problem is broken doc patches, then maybe a solution is to
> include the `xmllint --noout --valid` target in whatever the check-world
> equivalent is for meson.

+1, but let's do that for the "make" build too.  I see that
doc/src/sgml/Makefile has a "check" target, but AFAICS it's not
wired up to the top-level check-world.

			regards, tom lane

  32. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-11-07T17:00:11Z

    Hi,

On 2023-11-06 10:45:27 +0100, Christoph Berg wrote:
> Re: Andres Freund
> > > > The reason for that is simply that the docs take too long to build.
> > >
> > > That why I'd prefer to be able to separate arch:all and arch:any
> > > builds, yes.
> > 
> > What's stopping you from doing that?  I think the only arch:any content we
> > have is the docs, and those you can build separately? Doc builds do trigger
> > generation of a handful of files besides the docs, but not more.
> 
> Historically, .deb files have been required to contain the manpages
> for all executables even when there's a separate -doc package. This
> means we'd need a separate (hopefully fast) manpage build even when
> the arch:any binaries are built.

Manpages are a bit faster to build than html, but not a whole lot. Both are a
lot faster than PDF.


> We might get around that by introducing a new postgresql-manpages-XX
> arch:all package, but that might be too much micropackaging.

I've not done packaging in, uh, a fair while, but isn't the common solution to
that a -common package? There might be a few more files we could put itno one.


> > +     <varlistentry id="configure-with-sepgsql-meson">
> > +      <term><option>-Dselinux={ disabled | auto | enabled }</option></term>
> > +      <listitem>
> > +       <para>
> > +        Build with selinux support, enabling the <xref linkend="sepgsql"/>
> > +        extension.
> 
> This option defaults to ... auto?

Not quite sure what you mean? Today it defaults to disabled, a patch changing
that should also change the docs?


> > index 90e2c062fa8..003b57498bb 100644
> > --- a/doc/src/sgml/meson.build
> > +++ b/doc/src/sgml/meson.build
> > @@ -142,6 +142,7 @@ if docs_dep.found()
> >        '--install-dir-contents', dir_doc_html, html],
> >      build_always_stale: true, build_by_default: false,
> >    )
> > +  alias_target('doc-html', install_doc_html)
> >    alias_target('install-doc-html', install_doc_html)
> 
> Shouldn't this just build the html docs, without installing?
> 
> > +  alias_target('doc-man', install_doc_html)
> >    alias_target('install-doc-man', install_doc_man)
> 
> ... same
> 
> 
> > +     <varlistentry id="meson-target-install-world">
> > +       <term><option>install-install-world</option></term>
> 
> install-world
> 
> > +     <varlistentry id="meson-target-install-doc-man">
> > +       <term><option>install-doc-html</option></term>
> > +       <listitem>
> > +        <para>
> > +         Install documentation in man page format.
> 
> install-doc-man

Oops.


> > +   <sect3 id="meson-targets-docs">
> > +    <title>Documentation Targets</title>
> 
> > +     <varlistentry id="meson-target-docs">
> > +       <term><option>docs</option></term>
> > +       <term><option>doc-html</option></term>
> > +       <listitem>
> > +        <para>
> > +         Build documentation in multi-page HTML format.  Note that
> > +         <option>docs</option> does <emphasis>not</emphasis> include building
> > +         man page documentation, as man page generation seldom fails when
> > +         building HTML documentation succeeds.
> 
> Why is that a reason for not building the manpages?

I didn't have it that way, and Tom argued strongly for maintaining that
behaviour from the make build. Personally I wouldn't.



> > +   <sect3 id="meson-targets-code">
> > +    <title>Code Targets</title>
> 
> I would have expected the sections to be in the order
> build-docs-install. Having install first seems weird to me.

Makes sense to me. I just had the install first because I wrote it first
because of our conversation...


> > +   <sect3 id="meson-targets-other">
> > +    <title>Other Targets</title>
> > +
> > +    <variablelist>
> > +
> > +     <varlistentry id="meson-target-clean">
> > +       <term><option>clean</option></term>
> > +       <listitem>
> > +        <para>
> > +         Remove all build products
> > +        </para>
> > +       </listitem>
> > +     </varlistentry>
> > +
> > +     <varlistentry id="meson-target-test">
> > +       <term><option>test</option></term>
> > +       <listitem>
> > +        <para>
> > +         Remove all enabled tests. Support for some classes of tests can be
> > +         enabled / disabled with <xref linkend="configure-tap-tests-meson"/>
> > +         and <xref linkend="configure-pg-test-extra-meson"/>.
> 
> This should explicitly say if contrib tests are included (or there
> needs to be a separate test-world target.)

They are included, will state that. And also s/Remove/Run/


> > Subject: [PATCH v1 5/5] meson: Add -Dpkglibdir option
> 
> Will give that a try, thanks!

Thanks for the review!

Greetings,

Andres Freund

  33. Re: meson documentation build open issues

    Christoph Berg <myon@debian.org> — 2023-11-07T17:07:11Z

    Re: Andres Freund
> > We might get around that by introducing a new postgresql-manpages-XX
> > arch:all package, but that might be too much micropackaging.
> 
> I've not done packaging in, uh, a fair while, but isn't the common solution to
> that a -common package? There might be a few more files we could put itno one.

True. /usr/share/postgresql/17/ is 4.2MB here, with 1.5MB manpages,
1.1MB /extensions/ and some other bits. Will consider, thanks.

> > > +     <varlistentry id="configure-with-sepgsql-meson">
> > > +      <term><option>-Dselinux={ disabled | auto | enabled }</option></term>
> > > +      <listitem>
> > > +       <para>
> > > +        Build with selinux support, enabling the <xref linkend="sepgsql"/>
> > > +        extension.
> > 
> > This option defaults to ... auto?
> 
> Not quite sure what you mean? Today it defaults to disabled, a patch changing
> that should also change the docs?

What I failed to say is that the other options document what the
default it, this one doesn't yet.

Christoph

  34. Re: meson documentation build open issues

    Peter Eisentraut <peter@eisentraut.org> — 2023-11-08T11:04:30Z

    On 07.11.23 17:08, Andres Freund wrote:
>> make world
>> make check-world
>> make install-world
>>
>> that encompasses everything.
>>
>> Now with meson to handle the documentation one needs to remember a variety of additional targets.  (There is a risk that once this gets more widespread, more people will submit broken documentation.)
> install-world with meson also installs docs.

Ok, I didn't know about ninja install-world.  That works for me.  Maybe 
a "world" target would also be good.

I played around with this a bit and noticed some files missing or in the 
wrong place.  See two attached patches (plus e9f075f9a1 already committed).

  35. Re: meson documentation build open issues

    Peter Eisentraut <peter@eisentraut.org> — 2023-11-08T11:05:52Z

    On 07.11.23 17:40, Alvaro Herrera wrote:
> If the problem is broken doc patches, then maybe a solution is to
> include the `xmllint --noout --valid` target in whatever the check-world
> equivalent is for meson.  Looking at doc/src/sgml/meson.build, we don't
> seem to do that anywhere.  Doing the no-output lint run is very fast
> (375ms real time in my machine, whereas "make html" takes 27s).

This would be a start, but it wouldn't cover everything.  Lately, we 
require id attributes on certain elements, which is checked on the XSLT 
level.

  36. Re: meson documentation build open issues

    Christoph Berg <myon@debian.org> — 2023-11-08T12:55:02Z

    Re: Peter Eisentraut
> > If the problem is broken doc patches, then maybe a solution is to
> > include the `xmllint --noout --valid` target in whatever the check-world
> > equivalent is for meson.  Looking at doc/src/sgml/meson.build, we don't
> > seem to do that anywhere.  Doing the no-output lint run is very fast
> > (375ms real time in my machine, whereas "make html" takes 27s).
> 
> This would be a start, but it wouldn't cover everything.  Lately, we require
> id attributes on certain elements, which is checked on the XSLT level.

I'd think there should be a catchy "make check-world"-equivalent that
does run all reasonable check that we can tell people to run by
default. Then if that takes too long, we could still offer
alternatives that exclude some areas. If it's the other way round,
some areas will never be checked widely.

Christoph

  37. Re: meson documentation build open issues

    Peter Eisentraut <peter@eisentraut.org> — 2023-11-08T15:19:51Z

    On 08.11.23 13:55, Christoph Berg wrote:
> Re: Peter Eisentraut
>>> If the problem is broken doc patches, then maybe a solution is to
>>> include the `xmllint --noout --valid` target in whatever the check-world
>>> equivalent is for meson.  Looking at doc/src/sgml/meson.build, we don't
>>> seem to do that anywhere.  Doing the no-output lint run is very fast
>>> (375ms real time in my machine, whereas "make html" takes 27s).
>>
>> This would be a start, but it wouldn't cover everything.  Lately, we require
>> id attributes on certain elements, which is checked on the XSLT level.
> 
> I'd think there should be a catchy "make check-world"-equivalent that
> does run all reasonable check that we can tell people to run by
> default. Then if that takes too long, we could still offer
> alternatives that exclude some areas. If it's the other way round,
> some areas will never be checked widely.

I think we could build doc/src/sgml/postgres-full.xml by default.  That 
takes less than 0.5 seconds here and it's an intermediate target for 
html and man.

  38. Re: meson documentation build open issues

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2023-11-08T15:55:07Z

    On 2023-Nov-08, Peter Eisentraut wrote:

> I think we could build doc/src/sgml/postgres-full.xml by default.  That
> takes less than 0.5 seconds here and it's an intermediate target for html
> and man.

If that detects problems like the id attributes you mentioned, apart
from the other checks in the `xmllint --noout`, then that WFM.

At least with the makefile the command to produce postgres-full.xml
includes --valid, so I think we're covered.

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"Having your biases confirmed independently is how scientific progress is
made, and hence made our great society what it is today" (Mary Gardiner)

  39. Re: meson documentation build open issues

    Tristan Partin <tristan@neon.tech> — 2023-11-08T15:56:08Z

    Looks good to me. Thanks for finding this.

-- 
Tristan Partin
Neon (https://neon.tech)

  40. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-11-08T17:32:08Z

    Hi,

On 2023-11-08 12:04:30 +0100, Peter Eisentraut wrote:
> Ok, I didn't know about ninja install-world.  That works for me.  Maybe a
> "world" target would also be good.

Yea, I thought so as well. I'll send out a patch shortly. Kinda wondering if
its worth backpatching to 16. Uniformity seems useful and it's low risk.


> I played around with this a bit and noticed some files missing or in the
> wrong place.  See two attached patches (plus e9f075f9a1 already committed).

Make sense.

Greetings,

Andres Freund

  41. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-11-08T23:21:21Z

    Hi,

I really like the idea of an 'help' target that prints the targets. It seemed
annoying to document such targets in both the sgml docs and the input for a
the help target. Particularly due to the redundancies between id attributes,
the target name etc.

First I generated the list of targets from within meson.build, only to later
realize that that would not work when building the docs via make. So I instead
added doc/src/sgml/meson-targets.txt which is lightly postprocessed for the
'help' target, and slightly more processed when building the docs.

That does have some downsides, e.g. it'd be more complicated to only print
targets if a relevant option is enabled. But I think it's acceptable that way.


Example output:

$ ninja help
[0/1 1   0%] Running external command help (wrapped by meson to set env)
Code Targets:
  all                          Build everything other than documentation
  backend                      Build backend and related modules
  bin                          Build frontend binaries
  contrib                      Build contrib modules
  pl                           Build procedual languages

Documentation Targets:
  docs                         Build documentation in multi-page HTML format
  doc-html                     Build documentation in multi-page HTML format
  doc-man                      Build documentation in man page format
  doc/src/sgml/postgres-A4.pdf Build documentation in PDF format, with A4 pages
  doc/src/sgml/postgres-US.pdf Build documentation in PDF format, with US letter pages
  doc/src/sgml/postgres.html   Build documentation in single-page HTML format
  alldocs                      Build documentation in all supported formats

Installation Targets:
  install                      Install postgres, excluding documentation
  install-doc-html             Install documentation in multi-page HTML format
  install-doc-man              Install documentation in man page format
  install-docs                 Install documentation in multi-page HTML and man page formats
  install-quiet                Like "install", but installed files are not displayed
  install-world                Install postgres, including multi-page HTML and man page documentation
  uninstall                    Remove installed files

Other Targets:
  clean                        Remove all build products
  test                         Run all enabled tests (including contrib)
  world                        Build everything, including documentation
  help                         List important targets


Because of the common source, some of the descriptions in the state of this
patch are a bit shorter than in the preceding commit. But I don't think that
hurts much.

Greetings,

Andres Freund

  42. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-11-09T00:43:25Z

    Hi,

On 2023-11-08 13:55:02 +0100, Christoph Berg wrote:
> Re: Peter Eisentraut
> > > If the problem is broken doc patches, then maybe a solution is to
> > > include the `xmllint --noout --valid` target in whatever the check-world
> > > equivalent is for meson.  Looking at doc/src/sgml/meson.build, we don't
> > > seem to do that anywhere.  Doing the no-output lint run is very fast
> > > (375ms real time in my machine, whereas "make html" takes 27s).
> > 
> > This would be a start, but it wouldn't cover everything.  Lately, we require
> > id attributes on certain elements, which is checked on the XSLT level.
> 
> I'd think there should be a catchy "make check-world"-equivalent that
> does run all reasonable check that we can tell people to run by
> default. Then if that takes too long, we could still offer
> alternatives that exclude some areas. If it's the other way round,
> some areas will never be checked widely.

The 'test' target (generated by meson, otherwise I'd have named it check),
runs all enabled tests.  You obviously can run a subset if you so desire.

Greetings,

Andres Freund

  43. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-11-09T00:59:09Z

    Hi,

On 2023-11-08 16:19:51 +0100, Peter Eisentraut wrote:
> On 08.11.23 13:55, Christoph Berg wrote:
> > Re: Peter Eisentraut
> > > > If the problem is broken doc patches, then maybe a solution is to
> > > > include the `xmllint --noout --valid` target in whatever the check-world
> > > > equivalent is for meson.  Looking at doc/src/sgml/meson.build, we don't
> > > > seem to do that anywhere.  Doing the no-output lint run is very fast
> > > > (375ms real time in my machine, whereas "make html" takes 27s).
> > >
> > > This would be a start, but it wouldn't cover everything.  Lately, we require
> > > id attributes on certain elements, which is checked on the XSLT level.
> >
> > I'd think there should be a catchy "make check-world"-equivalent that
> > does run all reasonable check that we can tell people to run by
> > default. Then if that takes too long, we could still offer
> > alternatives that exclude some areas. If it's the other way round,
> > some areas will never be checked widely.
>
> I think we could build doc/src/sgml/postgres-full.xml by default.  That
> takes less than 0.5 seconds here and it's an intermediate target for html
> and man.

That does require the docbook dtd to be installed, afaict. I think we would
need a configure test for that to be present if we want to build it by
default, otherwise we'll cause errors on plenty systems that don't get them
today.  The docbook dts aren't a huge dependency, but still. Some OSs might
not have a particularly install source for them, e.g. windows.

I don't think that'd detect the missing ids?

Greetings,

Andres Freund

  44. Re: meson documentation build open issues

    Peter Eisentraut <peter@eisentraut.org> — 2023-11-09T14:32:39Z

    On 09.11.23 01:59, Andres Freund wrote:
>> I think we could build doc/src/sgml/postgres-full.xml by default.  That
>> takes less than 0.5 seconds here and it's an intermediate target for html
>> and man.
> That does require the docbook dtd to be installed, afaict. I think we would
> need a configure test for that to be present if we want to build it by
> default, otherwise we'll cause errors on plenty systems that don't get them
> today.  The docbook dts aren't a huge dependency, but still. Some OSs might
> not have a particularly install source for them, e.g. windows.

I was thinking we would do it only if the required tools are found. 
Basically like

  postgres_full_xml = custom_target('postgres-full.xml',
    input: 'postgres.sgml',
    output: 'postgres-full.xml',
    depfile: 'postgres-full.xml.d',
    command: [xmllint, '--nonet', '--noent', '--valid',
              '--path', '@OUTDIR@', '-o', '@OUTPUT@', '@INPUT@'],
    depends: doc_generated,
-  build_by_default: false,
+  build_by_default: xmllint_bin.found(),
  )

Besides giving you a quick validity check of the XML, this also builds 
the doc_generated, which draw from non-doc source files, so this would 
also serve to check that those are sound and didn't mess up the docs.

> I don't think that'd detect the missing ids?

Right, it wouldn't do that.

  45. Re: meson documentation build open issues

    Peter Eisentraut <peter@eisentraut.org> — 2023-11-09T14:39:32Z

    On 09.11.23 00:21, Andres Freund wrote:
> Example output:

This is very nice!

> $ ninja help
> [0/1 1   0%] Running external command help (wrapped by meson to set env)
> Code Targets:
>    all                          Build everything other than documentation
>    backend                      Build backend and related modules
>    bin                          Build frontend binaries
>    contrib                      Build contrib modules
>    pl                           Build procedual languages

ok

> Documentation Targets:
>    docs                         Build documentation in multi-page HTML format
>    doc-html                     Build documentation in multi-page HTML format
>    doc-man                      Build documentation in man page format
>    doc/src/sgml/postgres-A4.pdf Build documentation in PDF format, with A4 pages
>    doc/src/sgml/postgres-US.pdf Build documentation in PDF format, with US letter pages
>    doc/src/sgml/postgres.html   Build documentation in single-page HTML format
>    alldocs                      Build documentation in all supported formats
> 
> Installation Targets:
>    install                      Install postgres, excluding documentation

This should probably read "Install everything other than documentation", 
to mirror "all" above.  (Otherwise one might think it installs just the 
backend.)

>    install-doc-html             Install documentation in multi-page HTML format
>    install-doc-man              Install documentation in man page format
>    install-docs                 Install documentation in multi-page HTML and man page formats

There is a mismatch between "docs" and "install-docs".  (As was 
previously discussed, I'm in the camp that "docs" should be html + man.)

>    install-quiet                Like "install", but installed files are not displayed
>    install-world                Install postgres, including multi-page HTML and man page documentation

Suggest "Install everything, including documentation" (matches "world").

>    uninstall                    Remove installed files
> 
> Other Targets:
>    clean                        Remove all build products
>    test                         Run all enabled tests (including contrib)
>    world                        Build everything, including documentation

Shouldn't that be under "Code Targets"?

>    help                         List important targets

  46. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-11-09T17:52:40Z

    Hi,

On 2023-11-09 15:32:39 +0100, Peter Eisentraut wrote:
> On 09.11.23 01:59, Andres Freund wrote:
> > > I think we could build doc/src/sgml/postgres-full.xml by default.  That
> > > takes less than 0.5 seconds here and it's an intermediate target for html
> > > and man.
> > That does require the docbook dtd to be installed, afaict. I think we would
> > need a configure test for that to be present if we want to build it by
> > default, otherwise we'll cause errors on plenty systems that don't get them
> > today.  The docbook dts aren't a huge dependency, but still. Some OSs might
> > not have a particularly install source for them, e.g. windows.
> 
> I was thinking we would do it only if the required tools are found.
> Basically like
> 
>  postgres_full_xml = custom_target('postgres-full.xml',
>    input: 'postgres.sgml',
>    output: 'postgres-full.xml',
>    depfile: 'postgres-full.xml.d',
>    command: [xmllint, '--nonet', '--noent', '--valid',
>              '--path', '@OUTDIR@', '-o', '@OUTPUT@', '@INPUT@'],
>    depends: doc_generated,
> -  build_by_default: false,
> +  build_by_default: xmllint_bin.found(),
>  )

We don't get to that point if xmllint isn't found...


> Besides giving you a quick validity check of the XML, this also builds the
> doc_generated, which draw from non-doc source files, so this would also
> serve to check that those are sound and didn't mess up the docs.

Unfortunately presence of xmllint doesn't guarantee presence of the relevant
DTDs. Without docbook-xml installed, you'll get something like

../../../../../home/andres/src/postgresql/doc/src/sgml/postgres.sgml:21: warning: failed to load external entity "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"
]>
  ^

and a bunch of other subsequent errors.


I think if we want to do this, we'd need a configure time check for being able
to validate a document with
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"...

That's certainly doable. If we go there, we imo also should check if the
relevant xslt stylesheets are installed.

Greetings,

Andres Freund

  47. Re: meson documentation build open issues

    Peter Eisentraut <peter@eisentraut.org> — 2023-11-14T10:58:53Z

    On 09.11.23 18:52, Andres Freund wrote:
>> Besides giving you a quick validity check of the XML, this also builds the
>> doc_generated, which draw from non-doc source files, so this would also
>> serve to check that those are sound and didn't mess up the docs.
> Unfortunately presence of xmllint doesn't guarantee presence of the relevant
> DTDs. Without docbook-xml installed, you'll get something like
> 
> ../../../../../home/andres/src/postgresql/doc/src/sgml/postgres.sgml:21: warning: failed to load external entity"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"
> ]>
>    ^
> 
> and a bunch of other subsequent errors.
> 
> 
> I think if we want to do this, we'd need a configure time check for being able
> to validate a document with
> <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN""http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"...

We used to have exactly such a check in configure, but it was removed in 
4823c4f6ac.  I suppose we could look into reviving that.

  48. Re: meson documentation build open issues

    Peter Eisentraut <peter@eisentraut.org> — 2023-11-14T20:16:13Z

    Some comments on your patches:

v2-0001-meson-Change-default-of-selinux-feature-option-to.patch

Ok

v2-0002-docs-Document-with-selinux-Dselinux-options-centr.patch

Ok, but "selinux" should be "SELinux" when referring to the product.

v2-0003-meson-docs-Add-doc-html-man-targets.patch

We have make targets "html" and "man", so I suggest we make the meson 
targets the same.

v2-0004-meson-Add-world-target.patch

AFAICT, this world target doesn't include the man target.  (Again, this 
would all work better if we added "man" to "docs".)

v2-0005-docs-meson-Add-documentation-for-important-build-.patch

It's nice to document this, but it's weird that we only document the 
meson targets, not the make targets.

v2-0006-meson-Add-help-target-build-docs-from-a-common-so.patch

Here also, making this consistent and uniform with make would be useful.

v2-0007-meson-Add-Dpkglibdir-option.patch

Normally, the pkgFOOdir variables are just FOOdir plus package name.  I 
don't feel comfortable allowing those to be separately set.  We don't 
allow that with configure; this just arose from a Debian patch.

The description "location to dynamically loadable modules" is too 
narrow.  Consider for example, another proposed patch, where we are 
doing some preprocessing on postgres.bki at build time.  Since that 
makes postgres.bki platform dependent, it should really be moved from 
share (datadir in configure parlance) to pkglibdir.  So then we have 
things in there that are not loadable modules.  I don't know how that 
affects Debian packaging, but this patch might not be the right one.

I suggest we leave this patch for a separate discussion.

  49. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-11-15T00:22:31Z

    Hi,

On 2023-11-14 21:16:13 +0100, Peter Eisentraut wrote:
> Some comments on your patches:
> 
> v2-0001-meson-Change-default-of-selinux-feature-option-to.patch
> 
> Ok
> 
> v2-0002-docs-Document-with-selinux-Dselinux-options-centr.patch
> 
> Ok, but "selinux" should be "SELinux" when referring to the product.

Will apply with that fix.


> v2-0003-meson-docs-Add-doc-html-man-targets.patch
> 
> We have make targets "html" and "man", so I suggest we make the meson
> targets the same.

Hm, ok.


> v2-0004-meson-Add-world-target.patch
> 
> AFAICT, this world target doesn't include the man target.  (Again, this
> would all work better if we added "man" to "docs".)

I agree with that sentiment - I only moved to the current arrangement after
Tom argued forcefully against building both.

The situation in the make world is weird:
"make docs" in the toplevel builds both, because it's defined as

docs:
	$(MAKE) -C doc all

Buf if you "make -C doc/src/sgml" (or are in doc/src/sgml), we only build
html, as the default target is explicitly just html:

# Make "html" the default target, since that is what most people tend
# to want to use.
html:


There's no real way of making the recursive-make and non-recursive ninja
coherent. There's no equivalent to default target in a sudirectory with ninja
(or non-recursive make).


> v2-0005-docs-meson-Add-documentation-for-important-build-.patch
> 
> It's nice to document this, but it's weird that we only document the meson
> targets, not the make targets.

I think it'd have been good if we had documented the important targets with
make. But I don't think documenting them as a prerequisite to documenting the
meson targets makes much sense.


> v2-0006-meson-Add-help-target-build-docs-from-a-common-so.patch
> 
> Here also, making this consistent and uniform with make would be useful.

What precisely are you referring to here? Also adding a help target? Or just
consistency between what the "docs" target does?


> v2-0007-meson-Add-Dpkglibdir-option.patch
> 
> Normally, the pkgFOOdir variables are just FOOdir plus package name.  I
> don't feel comfortable allowing those to be separately set.  We don't allow
> that with configure; this just arose from a Debian patch.

Right - but Debian's desire seems quite sensible. The need to have multiple
postgres versions installed in parallel is quite widespread.


> The description "location to dynamically loadable modules" is too narrow.
> Consider for example, another proposed patch, where we are doing some
> preprocessing on postgres.bki at build time.  Since that makes postgres.bki
> platform dependent, it should really be moved from share (datadir in
> configure parlance) to pkglibdir.

I think I cannot be faulted for documenting the current use of the directory
:).

Separately, I'm not really convinced that moving some build time values into
postgres.bki is useful, but that's a matter for a different thread.


> So then we have things in there that are not loadable modules.  I don't know
> how that affects Debian packaging, but this patch might not be the right
> one.

I'm not really seeing why that'd affect pkglibdir being adjustable, besides
needing to tweak the description of pkglibdir?


> I suggest we leave this patch for a separate discussion.

Fair enough.

Thanks for the review,

Andres Freund

  50. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-11-15T00:26:25Z

    Hi,

On 2023-11-14 11:58:53 +0100, Peter Eisentraut wrote:
> On 09.11.23 18:52, Andres Freund wrote:
> > I think if we want to do this, we'd need a configure time check for being able
> > to validate a document with
> > <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN""http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"...
> 
> We used to have exactly such a check in configure, but it was removed in
> 4823c4f6ac.  I suppose we could look into reviving that.

Yea, that change was obsoleted by xmllint/xsltproc not being able to fetch the
dtd over network anymore... And the performance issue 4823c4f6ac notes also
doesn't apply anymore, as we use -nonet since 969509c3f2e.

Greetings,

Andres Freund

  51. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-11-15T00:30:24Z

    Hi,

On 2023-11-14 16:22:31 -0800, Andres Freund wrote:
> > v2-0004-meson-Add-world-target.patch
> > 
> > AFAICT, this world target doesn't include the man target.  (Again, this
> > would all work better if we added "man" to "docs".)
> 
> I agree with that sentiment - I only moved to the current arrangement after
> Tom argued forcefully against building both.

Another message in this thread made me realize that I actually hadn't
implemented it at all - it was Tom in 969509c3f2e

    In HEAD, also document how to build docs using Meson, and adjust
    "ninja docs" to just build the HTML docs, for consistency with the
    default behavior of doc/src/sgml/Makefile.


I think that change was just ill-advised, given that the top-level make target
actually *does* build both html and man:

> The situation in the make world is weird:
> "make docs" in the toplevel builds both, because it's defined as
> 
> docs:
> 	$(MAKE) -C doc all

Notwithstanding this:

> Buf if you "make -C doc/src/sgml" (or are in doc/src/sgml), we only build
> html, as the default target is explicitly just html:

As the obvious thing for people that really just want to build html with ninja
would be to just use the doc-html (to-be-renamed to "html") target.

Greetings,

Andres Freund

  52. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-11-17T18:53:06Z

    Hi,

On 2023-11-14 16:30:24 -0800, Andres Freund wrote:
> On 2023-11-14 16:22:31 -0800, Andres Freund wrote:
> > > v2-0004-meson-Add-world-target.patch
> > > 
> > > AFAICT, this world target doesn't include the man target.  (Again, this
> > > would all work better if we added "man" to "docs".)
> > 
> > I agree with that sentiment - I only moved to the current arrangement after
> > Tom argued forcefully against building both.
> 
> Another message in this thread made me realize that I actually hadn't
> implemented it at all - it was Tom in 969509c3f2e
> 
>     In HEAD, also document how to build docs using Meson, and adjust
>     "ninja docs" to just build the HTML docs, for consistency with the
>     default behavior of doc/src/sgml/Makefile.
> 
> 
> I think that change was just ill-advised, given that the top-level make target
> actually *does* build both html and man:
> 
> > The situation in the make world is weird:
> > "make docs" in the toplevel builds both, because it's defined as
> > 
> > docs:
> > 	$(MAKE) -C doc all
> 
> Notwithstanding this:
> 
> > Buf if you "make -C doc/src/sgml" (or are in doc/src/sgml), we only build
> > html, as the default target is explicitly just html:
> 
> As the obvious thing for people that really just want to build html with ninja
> would be to just use the doc-html (to-be-renamed to "html") target.

I pushed the first two commits (the selinux stuff) and worked a bit more on
the subsequent ones.

- As requested, I've renamed the 'doc-html' and 'doc-man' targets to just 'html'
  and 'man'. Which then seems to also necessitates renaming the existing
  install-doc-{html,man}.  I'm not so sure about this change, likely because I
  use autocomplete to remember the spelling of ninja (or make) targets, which
  is easier with [install-]doc-{html,man} than with [install-]{html,man}.

- I added a commit to change what 'docs' builds, undoing that part of
  969509c3f2e. I also moved the 'all' target in doc/src/sgml/Makefile up to
  the 'html' target to make things less confusing there, as discussed in the
  thread referenced in the commit message.

  Because of the 'html' target, Tom can still just build html easily.

- I renamed 'meson-targets.txt' to 'targets-meson.txt' and renamed other files
  to match. One reason is that meson tries to prevent conflict between its
  internal targets by prefixing them with 'meson-', and the old names
  conflicted with that rule. If we ever wanted to add something similar for
  make, the new naming also seems better.

- I added documentation for some developer targets (reformat-dat-files,
  expand-dat-files, update-unicode)

I didn't move 'world' in the docs, as it doesn't quite seem right in the "code
targets" section?


I attached the pkglibdir thing again, even though I don't plan to push it or
really review it further. Thought it might still be interesting for Christoph.

Greetings,

Andres Freund

  53. Re: meson documentation build open issues

    Peter Eisentraut <peter@eisentraut.org> — 2023-11-20T07:27:48Z

    On 17.11.23 19:53, Andres Freund wrote:
> I pushed the first two commits (the selinux stuff) and worked a bit more on
> the subsequent ones.

Patches 0001 through 0004 look good to me.

Some possible small tweaks in 0004:

+    perl, '-ne', 'next if /^#/; print',

If you're going for super-brief mode, you could also use "perl -p" and 
drop the "print".

Put at least two spaces between the "columns" in targets-meson.txt:

+  doc/src/sgml/postgres-A4.pdf  Build documentation in PDF format, with
                                ^^

  54. Re: meson documentation build open issues

    Andres Freund <andres@anarazel.de> — 2023-11-21T01:56:13Z

    Hi,

On 2023-11-20 08:27:48 +0100, Peter Eisentraut wrote:
> On 17.11.23 19:53, Andres Freund wrote:
> > I pushed the first two commits (the selinux stuff) and worked a bit more on
> > the subsequent ones.
> 
> Patches 0001 through 0004 look good to me.

Cool, I pushed them now.


> Some possible small tweaks in 0004:
> 
> +    perl, '-ne', 'next if /^#/; print',
> 
> If you're going for super-brief mode, you could also use "perl -p" and drop
> the "print".

I thought this didn't add much, so I didn't go there.


> Put at least two spaces between the "columns" in targets-meson.txt:
> 
> +  doc/src/sgml/postgres-A4.pdf  Build documentation in PDF format, with
>                                ^^

I did adopt this.


One remaining question is whether we should adjust install-doc-{html,man} to
be install-{html,man}, to match the docs targets.

Greetings,

Andres Freund

  55. Re: meson documentation build open issues

    Peter Eisentraut <peter@eisentraut.org> — 2023-11-21T13:23:39Z

    On 21.11.23 02:56, Andres Freund wrote:
> One remaining question is whether we should adjust install-doc-{html,man} to
> be install-{html,man}, to match the docs targets.

Ah didn't notice that one; yes please.

  56. Re: meson documentation build open issues

    Peter Eisentraut <peter@eisentraut.org> — 2023-11-21T14:54:10Z

    On 21.11.23 14:23, Peter Eisentraut wrote:
> On 21.11.23 02:56, Andres Freund wrote:
>> One remaining question is whether we should adjust 
>> install-doc-{html,man} to
>> be install-{html,man}, to match the docs targets.
> 
> Ah didn't notice that one; yes please.

I think this was done?