Horton

Thread

Commits

  1. Add new make targets world-bin and install-world-bin

  1. make world and install-world without docs

    Andrew Dunstan <andrew@dunslane.net> — 2021-05-31T14:16:11Z

    I've been thinking about rationalizing some of the buildfarm code, which
has grown somewhat like Topsy over the years. One useful thing would be
to run all the "make" and "install" pieces together. When the buildfarm
started we didn't have world targets, but they are now almost ancient
history themselves, so it would be nice to leverage them.

However, not all buildfarm animals are set up to build the docs, and not
all owners necessarily want to. Moreover, we have provision for testing
various docs formats (PDF, epub etc). So I'd like to be able to build
and install all the world EXCEPT the docs. Rather than specify yet more
targets in the Makefile, it seemed to me a better way would be to
provide a SKIPDOCS option that could be set on the command line like this:

    make SKIPDOCS=1 world
    make SKIPDOCS=1 install-world

The attached very small patch is intended to provide for that.
Incidentally, this is exactly what the MSVC build system's 'build.bat'
and 'install.bat' do.

I should add that quite apart from the buildfarm considerations this is
something I've long wanted, and I suspect other developers would find it
useful too.

Obviously to be useful to the buildfarm it would need to be backpatched.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  2. Re: make world and install-world without docs

    Joel Jacobson <joel@compiler.org> — 2021-05-31T19:32:34Z

    On Mon, May 31, 2021, at 16:16, Andrew Dunstan wrote:
> However, not all buildfarm animals are set up to build the docs, and not
> all owners necessarily want to. Moreover, we have provision for testing
> various docs formats (PDF, epub etc). So I'd like to be able to build
> and install all the world EXCEPT the docs.

Why would someone not always want to test building the docs?
What makes the docs special?

/Joel

  3. Re: make world and install-world without docs

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-05-31T20:07:19Z

    "Joel Jacobson" <joel@compiler.org> writes:
> On Mon, May 31, 2021, at 16:16, Andrew Dunstan wrote:
>> However, not all buildfarm animals are set up to build the docs, and not
>> all owners necessarily want to. Moreover, we have provision for testing
>> various docs formats (PDF, epub etc). So I'd like to be able to build
>> and install all the world EXCEPT the docs.

> Why would someone not always want to test building the docs?
> What makes the docs special?

Toolchain requirements, cf [1].  Per Andrew's comment, requiring all
that stuff to be installed would move the goalposts quite a ways for
buildfarm owners, and not all of the older systems we have in the farm
would be able to do it easily.  (If you don't have access to prebuilt
packages, you're looking at a lot of work to get that stuff
installed.)

It was a good deal worse when we used the TeX-based toolchain
to make PDFs, but it's still not something I want to foist on
buildfarm owners.  Especially since there's no real reason
to think that there are platform dependencies that would make
it valuable to run such builds on a spectrum of machines.
We do have a couple of machines that have opted-in to building
the docs, and that seems sufficient.  I feel no urge to make
it be opt-out instead.

			regards, tom lane

[1] https://www.postgresql.org/docs/devel/docguide-toolsets.html

  4. Re: make world and install-world without docs

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2021-06-01T00:06:46Z

    On 2021-May-31, Andrew Dunstan wrote:

> However, not all buildfarm animals are set up to build the docs, and not
> all owners necessarily want to. Moreover, we have provision for testing
> various docs formats (PDF, epub etc). So I'd like to be able to build
> and install all the world EXCEPT the docs. Rather than specify yet more
> targets in the Makefile, it seemed to me a better way would be to
> provide a SKIPDOCS option that could be set on the command line like this:
> 
>     make SKIPDOCS=1 world
>     make SKIPDOCS=1 install-world

I could use this feature.  +1

> +ifndef SKIPDOCS
>  $(call recurse,world,doc src config contrib,all)
>  world:
>  	+@echo "PostgreSQL, contrib, and documentation successfully made. Ready to install."
> +else
> +$(call recurse,world,src config contrib,all)
> +world:
> +	+@echo "PostgreSQL and contrib successfully made. Ready to install."
> +endif

I was going to suggest that instead of repeating the $(call) line you
could do something like

$(call recurse,world,src config contrib,all)
ifndef SKIPDOCS
$(call recurse,world,doc,all)
endif

... however, this makes the echoed string be wrong, and the whole thing
looks uglier if you use a second "ifndef" to generate the string, so I
think your proposal is okay.

I do wonder if these echoed strings are really all that necessary.

-- 
Álvaro Herrera                            39°49'30"S 73°17'W

  5. Re: make world and install-world without docs

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2021-06-01T22:20:04Z

    On 31.05.21 16:16, Andrew Dunstan wrote:
>      make SKIPDOCS=1 world
>      make SKIPDOCS=1 install-world

Maybe this should be configure option?  That's generally where you set 
what you want to build or not build.  (That might also make the 
buildfarm integration easier, since there are already facilities to 
specify and report configure options.)

  6. Re: make world and install-world without docs

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-06-01T22:23:32Z

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
> On 31.05.21 16:16, Andrew Dunstan wrote:
>> make SKIPDOCS=1 world
>> make SKIPDOCS=1 install-world

> Maybe this should be configure option?  That's generally where you set 
> what you want to build or not build.  (That might also make the 
> buildfarm integration easier, since there are already facilities to 
> specify and report configure options.)

Hmm, I think I prefer Andrew's way.  The fact that I don't want
to build the docs right now doesn't mean I won't want to do so
later --- in fact, that sequence is pretty exactly what I do
whenever I'm working on a patch.  It'd be annoying to have
to re-configure to make that work.

			regards, tom lane

  7. Re: make world and install-world without docs

    Andrew Dunstan <andrew@dunslane.net> — 2021-06-02T19:47:41Z

    On 6/1/21 6:23 PM, Tom Lane wrote:
> Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
>> On 31.05.21 16:16, Andrew Dunstan wrote:
>>> make SKIPDOCS=1 world
>>> make SKIPDOCS=1 install-world
>> Maybe this should be configure option?  That's generally where you set 
>> what you want to build or not build.  (That might also make the 
>> buildfarm integration easier, since there are already facilities to 
>> specify and report configure options.)
> Hmm, I think I prefer Andrew's way.  The fact that I don't want
> to build the docs right now doesn't mean I won't want to do so
> later --- in fact, that sequence is pretty exactly what I do
> whenever I'm working on a patch.  It'd be annoying to have
> to re-configure to make that work.
>
> 			



Yes, agreed. If you don't like the SKIPDOCS=1 mechanism, let's just
invent a couple of new targets instead, say `world-bin` and
`install-world-bin`.


I'm inclined to agree with Alvaro that the messages are at best an
oddity. Standard Unix practice is to be silent on success.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  8. Re: make world and install-world without docs

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-06-02T20:21:30Z

    Andrew Dunstan <andrew@dunslane.net> writes:
> I'm inclined to agree with Alvaro that the messages are at best an
> oddity. Standard Unix practice is to be silent on success.

We've been steadily moving towards less chatter during builds.
I'd be good with dropping these messages in HEAD, but doing so
in the back branches might be inadvisable.

			regards, tom lane

  9. Re: make world and install-world without docs

    Andrew Dunstan <andrew@dunslane.net> — 2021-07-01T14:47:07Z

    On 6/2/21 4:21 PM, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>> I'm inclined to agree with Alvaro that the messages are at best an
>> oddity. Standard Unix practice is to be silent on success.
> We've been steadily moving towards less chatter during builds.
> I'd be good with dropping these messages in HEAD, but doing so
> in the back branches might be inadvisable.
>
> 		



OK, I think on reflection new targets will be cleaner. What I suggest is
the attached, applied to all branches, followed by removal of the four
noise messages in just HEAD.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  10. Re: make world and install-world without docs

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-07-01T14:50:59Z

    Andrew Dunstan <andrew@dunslane.net> writes:
> OK, I think on reflection new targets will be cleaner. What I suggest is
> the attached, applied to all branches, followed by removal of the four
> noise messages in just HEAD.

Shouldn't these new targets be documented somewhere?

			regards, tom lane

  11. Re: make world and install-world without docs

    Andrew Dunstan <andrew@dunslane.net> — 2021-07-01T15:23:42Z

    On 7/1/21 10:50 AM, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>> OK, I think on reflection new targets will be cleaner. What I suggest is
>> the attached, applied to all branches, followed by removal of the four
>> noise messages in just HEAD.
> Shouldn't these new targets be documented somewhere?
>
> 			



Good point. See attached.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  12. Re: make world and install-world without docs

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-07-01T15:46:22Z

    Andrew Dunstan <andrew@dunslane.net> writes:
>> Shouldn't these new targets be documented somewhere?

> Good point. See attached.

+1, but spell check: "documantation"

			regards, tom lane

  13. Re: make world and install-world without docs

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2021-07-01T19:39:16Z

    On 01.07.21 16:47, Andrew Dunstan wrote:
> 
> On 6/2/21 4:21 PM, Tom Lane wrote:
>> Andrew Dunstan <andrew@dunslane.net> writes:
>>> I'm inclined to agree with Alvaro that the messages are at best an
>>> oddity. Standard Unix practice is to be silent on success.
>> We've been steadily moving towards less chatter during builds.
>> I'd be good with dropping these messages in HEAD, but doing so
>> in the back branches might be inadvisable.

> OK, I think on reflection new targets will be cleaner. What I suggest is
> the attached, applied to all branches, followed by removal of the four
> noise messages in just HEAD.

This naming approach is a bit problematic.  For example, we have 
"install-bin" in src/backend/, which is specifically for only installing 
binaries, not data files etc. (hence the name).  Your proposal would 
confuse this scheme.

I think we should also take a step back here and consider: We had "all", 
which wasn't "all" enough, then we had "world", now we have 
"world-minus-a-bit", but it's still more than "all".  It's like we are 
trying to prove the continuum hypothesis here.

I think we had consensus on the make variable approach, so I'm confused 
why a different solution was committed and backpatched without discussion.

  14. Re: make world and install-world without docs

    Andrew Dunstan <andrew@dunslane.net> — 2021-07-01T20:22:34Z

    On 7/1/21 3:39 PM, Peter Eisentraut wrote:
> On 01.07.21 16:47, Andrew Dunstan wrote:
>>
>> On 6/2/21 4:21 PM, Tom Lane wrote:
>>> Andrew Dunstan <andrew@dunslane.net> writes:
>>>> I'm inclined to agree with Alvaro that the messages are at best an
>>>> oddity. Standard Unix practice is to be silent on success.
>>> We've been steadily moving towards less chatter during builds.
>>> I'd be good with dropping these messages in HEAD, but doing so
>>> in the back branches might be inadvisable.
>
>> OK, I think on reflection new targets will be cleaner. What I suggest is
>> the attached, applied to all branches, followed by removal of the four
>> noise messages in just HEAD.
>
> This naming approach is a bit problematic.  For example, we have
> "install-bin" in src/backend/, which is specifically for only
> installing binaries, not data files etc. (hence the name).  Your
> proposal would confuse this scheme.
>
> I think we should also take a step back here and consider: We had
> "all", which wasn't "all" enough, then we had "world", now we have
> "world-minus-a-bit", but it's still more than "all".  It's like we are
> trying to prove the continuum hypothesis here.
>
> I think we had consensus on the make variable approach, so I'm
> confused why a different solution was committed and backpatched
> without discussion.


In fact the names and approach were suggested in my email of June 21st.

The make variable approach just felt klunky in the end.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  15. Re: make world and install-world without docs

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2021-07-01T20:29:49Z

    On 01.07.21 22:22, Andrew Dunstan wrote:
> 
> On 7/1/21 3:39 PM, Peter Eisentraut wrote:
>> On 01.07.21 16:47, Andrew Dunstan wrote:
>>>
>>> On 6/2/21 4:21 PM, Tom Lane wrote:
>>>> Andrew Dunstan <andrew@dunslane.net> writes:
>>>>> I'm inclined to agree with Alvaro that the messages are at best an
>>>>> oddity. Standard Unix practice is to be silent on success.
>>>> We've been steadily moving towards less chatter during builds.
>>>> I'd be good with dropping these messages in HEAD, but doing so
>>>> in the back branches might be inadvisable.
>>
>>> OK, I think on reflection new targets will be cleaner. What I suggest is
>>> the attached, applied to all branches, followed by removal of the four
>>> noise messages in just HEAD.
>>
>> This naming approach is a bit problematic.  For example, we have
>> "install-bin" in src/backend/, which is specifically for only
>> installing binaries, not data files etc. (hence the name).  Your
>> proposal would confuse this scheme.
>>
>> I think we should also take a step back here and consider: We had
>> "all", which wasn't "all" enough, then we had "world", now we have
>> "world-minus-a-bit", but it's still more than "all".  It's like we are
>> trying to prove the continuum hypothesis here.
>>
>> I think we had consensus on the make variable approach, so I'm
>> confused why a different solution was committed and backpatched
>> without discussion.
> 
> 
> In fact the names and approach were suggested in my email of June 21st.

AFAICT this thread contains no email from June 21st or thereabouts.

https://www.postgresql.org/message-id/flat/6a421136-d462-b043-a8eb-e75b2861f3df%40dunslane.net

  16. Re: make world and install-world without docs

    Andrew Dunstan <andrew@dunslane.net> — 2021-07-01T21:08:50Z

    On 7/1/21 4:29 PM, Peter Eisentraut wrote:
> On 01.07.21 22:22, Andrew Dunstan wrote:
>>
>> On 7/1/21 3:39 PM, Peter Eisentraut wrote:
>>> On 01.07.21 16:47, Andrew Dunstan wrote:
>>>>
>>>> On 6/2/21 4:21 PM, Tom Lane wrote:
>>>>> Andrew Dunstan <andrew@dunslane.net> writes:
>>>>>> I'm inclined to agree with Alvaro that the messages are at best an
>>>>>> oddity. Standard Unix practice is to be silent on success.
>>>>> We've been steadily moving towards less chatter during builds.
>>>>> I'd be good with dropping these messages in HEAD, but doing so
>>>>> in the back branches might be inadvisable.
>>>
>>>> OK, I think on reflection new targets will be cleaner. What I
>>>> suggest is
>>>> the attached, applied to all branches, followed by removal of the four
>>>> noise messages in just HEAD.
>>>
>>> This naming approach is a bit problematic.  For example, we have
>>> "install-bin" in src/backend/, which is specifically for only
>>> installing binaries, not data files etc. (hence the name).  Your
>>> proposal would confuse this scheme.
>>>
>>> I think we should also take a step back here and consider: We had
>>> "all", which wasn't "all" enough, then we had "world", now we have
>>> "world-minus-a-bit", but it's still more than "all".  It's like we are
>>> trying to prove the continuum hypothesis here.
>>>
>>> I think we had consensus on the make variable approach, so I'm
>>> confused why a different solution was committed and backpatched
>>> without discussion.
>>
>>
>> In fact the names and approach were suggested in my email of June 21st.
>
> AFAICT this thread contains no email from June 21st or thereabouts.
>
> https://www.postgresql.org/message-id/flat/6a421136-d462-b043-a8eb-e75b2861f3df%40dunslane.net
>


Apologies. June 2nd. One day American style dates will stop playing
havoc with my head - it's only been 25 years or so.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com