Horton

Thread

Commits

  1. Tweak generation of Gen_dummy_probes.pl

  2. Add a README and Makefile recipe for Gen_dummy_probes.pl

  3. Remove dependency on psed for MSVC builds.

  1. Why do we have perl and sed versions of Gen_dummy_probes?

    Andres Freund <andres@anarazel.de> — 2021-05-06T03:56:02Z

    Hi,

I wanted to apply https://postgr.es/m/CAGRY4nwaiPJc8wO0G7WZCgBmATC3GJVgvBoADZHDbCzhj8zTPw@mail.gmail.com
and noticed that there's not just Gen_dummy_probes.sed but also a
Gen_dummy_probes.pl.

I understand why we don't want to rely on sed because of windows - but
it's far from obvious why we can't just use the .pl variant all the
time?

The perl version was introduced in

commit 5d0320105699c253fe19b8b42ae1bffb67785b02
Author: Andrew Dunstan <andrew@dunslane.net>
Date:   2016-03-19 18:36:35 -0400

    Remove dependency on psed for MSVC builds.

    Modern Perl has removed psed from its core distribution, so it might not
    be readily available on some build platforms. We therefore replace its
    use with a Perl script generated by s2p, which is equivalent to the sed
    script. The latter is retained for non-MSVC builds to avoid creating a
    new hard dependency on Perl for non-Windows tarball builds.

    Backpatch to all live branches.

    Michael Paquier and me.

Greetings,

Andres Freund

  2. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-05-06T04:18:12Z

    Andres Freund <andres@anarazel.de> writes:
> I understand why we don't want to rely on sed because of windows - but
> it's far from obvious why we can't just use the .pl variant all the
> time?

Perl is not considered a hard build requirement on non-Windows.
We could dodge that by shipping a pre-built dummy probes.h,
but that doesn't really seem like a cleaner way than what's
there now.

Also, as I read it, Gen_dummy_probes.sed is useful in any case as
being the "source code" for Gen_dummy_probes.pl.  You'd need some
other form of documentation if you removed it.

			regards, tom lane

  3. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andres Freund <andres@anarazel.de> — 2021-05-06T04:59:14Z

    Hi,

On 2021-05-06 00:18:12 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > I understand why we don't want to rely on sed because of windows - but
> > it's far from obvious why we can't just use the .pl variant all the
> > time?
> 
> Perl is not considered a hard build requirement on non-Windows.

Oops, forgot that.


> We could dodge that by shipping a pre-built dummy probes.h,
> but that doesn't really seem like a cleaner way than what's
> there now.

I tried to regenerate Gen_dummy_probes.pl using s2p - which doesn't seem
to exist for modern versions of perl anymore :(


> Also, as I read it, Gen_dummy_probes.sed is useful in any case as
> being the "source code" for Gen_dummy_probes.pl.  You'd need some
> other form of documentation if you removed it.

:/

Greetings,

Andres Freund

  4. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> — 2021-05-06T10:13:28Z

    Andres Freund <andres@anarazel.de> writes:

> I tried to regenerate Gen_dummy_probes.pl using s2p - which doesn't seem
> to exist for modern versions of perl anymore :(

It still exists, it's just not part of the core Perl distribution any
more (since 5.22, released in 2015):

  https://metacpan.org/pod/perl5220delta#find2perl,-s2p-and-a2p-removal
  https://metacpan.org/release/App-s2p.

You can install it with `cpan App::s2p`.


- ilmari
-- 
"The surreality of the universe tends towards a maximum" -- Skud's Law
"Never formulate a law or axiom that you're not prepared to live with
 the consequences of."                              -- Skud's Meta-Law

  5. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andrew Dunstan <andrew@dunslane.net> — 2021-05-06T13:55:34Z

    On 5/6/21 12:59 AM, Andres Freund wrote:
> Hi,
>
> On 2021-05-06 00:18:12 -0400, Tom Lane wrote:
>> Andres Freund <andres@anarazel.de> writes:
>>> I understand why we don't want to rely on sed because of windows - but
>>> it's far from obvious why we can't just use the .pl variant all the
>>> time?
>> Perl is not considered a hard build requirement on non-Windows.
> Oops, forgot that.
>
>
>> We could dodge that by shipping a pre-built dummy probes.h,
>> but that doesn't really seem like a cleaner way than what's
>> there now.
> I tried to regenerate Gen_dummy_probes.pl using s2p - which doesn't seem
> to exist for modern versions of perl anymore :(
>
>
>> Also, as I read it, Gen_dummy_probes.sed is useful in any case as
>> being the "source code" for Gen_dummy_probes.pl.  You'd need some
>> other form of documentation if you removed it.


I suggest we add a README that sets out


a) why we do things this way

b) that the sed script is what's authoritative

c) how to regenerate the perl script if you change the sed script,
including where to get s2p


I can do that.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  6. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andrew Dunstan <andrew@dunslane.net> — 2021-05-07T15:19:02Z

    On 5/6/21 9:55 AM, Andrew Dunstan wrote:
> On 5/6/21 12:59 AM, Andres Freund wrote:
>> Hi,
>>
>> On 2021-05-06 00:18:12 -0400, Tom Lane wrote:
>>> Andres Freund <andres@anarazel.de> writes:
>>>> I understand why we don't want to rely on sed because of windows - but
>>>> it's far from obvious why we can't just use the .pl variant all the
>>>> time?
>>> Perl is not considered a hard build requirement on non-Windows.
>> Oops, forgot that.
>>
>>
>>> We could dodge that by shipping a pre-built dummy probes.h,
>>> but that doesn't really seem like a cleaner way than what's
>>> there now.
>> I tried to regenerate Gen_dummy_probes.pl using s2p - which doesn't seem
>> to exist for modern versions of perl anymore :(
>>
>>
>>> Also, as I read it, Gen_dummy_probes.sed is useful in any case as
>>> being the "source code" for Gen_dummy_probes.pl.  You'd need some
>>> other form of documentation if you removed it.
>
> I suggest we add a README that sets out
>
>
> a) why we do things this way
>
> b) that the sed script is what's authoritative
>
> c) how to regenerate the perl script if you change the sed script,
> including where to get s2p
>
>
> I can do that.
>
>


Here's a patch that adds the README and also adds a Makefile recipe for
regenerating Gen_dummy_probes.pl after the sed script is changed. On my
system at least the recipe is idempotent.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  7. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-05-07T17:04:25Z

    Andrew Dunstan <andrew@dunslane.net> writes:
> Here's a patch that adds the README and also adds a Makefile recipe for
> regenerating Gen_dummy_probes.pl after the sed script is changed. On my
> system at least the recipe is idempotent.

I've not tested the Makefile recipe, but the README looks good.

			regards, tom lane

  8. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andres Freund <andres@anarazel.de> — 2021-05-07T17:20:32Z

    Hi,

On 2021-05-07 11:19:02 -0400, Andrew Dunstan wrote:
> Here's a patch that adds the README and also adds a Makefile recipe for
> regenerating Gen_dummy_probes.pl after the sed script is changed. On my
> system at least the recipe is idempotent.

Nice! Thanks for this work.

Greetings,

Andres Freund

  9. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andres Freund <andres@anarazel.de> — 2021-05-07T17:36:32Z

    Hi,

On 2021-05-06 11:13:28 +0100, Dagfinn Ilmari Mannsåker wrote:
> Andres Freund <andres@anarazel.de> writes:
> 
> > I tried to regenerate Gen_dummy_probes.pl using s2p - which doesn't seem
> > to exist for modern versions of perl anymore :(
> 
> It still exists, it's just not part of the core Perl distribution any
> more (since 5.22, released in 2015):
> 
>   https://metacpan.org/pod/perl5220delta#find2perl,-s2p-and-a2p-removal
>   https://metacpan.org/release/App-s2p.

Oh, I got confused because the cpan link at the top of
https://perldoc.perl.org/5.6.2/s2p is dead, and because I forgot all I
knew about perl a long time ago.

Greetings,

Andres Freund

  10. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andrew Dunstan <andrew@dunslane.net> — 2021-05-07T18:31:43Z

    On 5/7/21 1:20 PM, Andres Freund wrote:
> Hi,
>
> On 2021-05-07 11:19:02 -0400, Andrew Dunstan wrote:
>> Here's a patch that adds the README and also adds a Makefile recipe for
>> regenerating Gen_dummy_probes.pl after the sed script is changed. On my
>> system at least the recipe is idempotent.
> Nice! Thanks for this work.
>


de nada. pushed.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  11. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2021-05-10T09:44:44Z

    On 07.05.21 20:31, Andrew Dunstan wrote:
> On 5/7/21 1:20 PM, Andres Freund wrote:
>> On 2021-05-07 11:19:02 -0400, Andrew Dunstan wrote:
>>> Here's a patch that adds the README and also adds a Makefile recipe for
>>> regenerating Gen_dummy_probes.pl after the sed script is changed. On my
>>> system at least the recipe is idempotent.
>> Nice! Thanks for this work.
> 
> de nada. pushed.

This recipe doesn't produce a Gen_dummy_probes.pl that matches exactly 
the one that is there now.  If this is going to be the preferred method, 
then we should generate it once so that it matches going forward.

  12. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> — 2021-05-10T11:16:12Z

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:

> On 07.05.21 20:31, Andrew Dunstan wrote:
>> On 5/7/21 1:20 PM, Andres Freund wrote:
>>> On 2021-05-07 11:19:02 -0400, Andrew Dunstan wrote:
>>>> Here's a patch that adds the README and also adds a Makefile recipe for
>>>> regenerating Gen_dummy_probes.pl after the sed script is changed. On my
>>>> system at least the recipe is idempotent.
>>> Nice! Thanks for this work.
>>
>> de nada. pushed.
>
> This recipe doesn't produce a Gen_dummy_probes.pl that matches exactly
> the one that is there now.  If this is going to be the preferred method,
> then we should generate it once so that it matches going forward.

Which version of perltidy do you have installed?  For me it generates
identical versions using any of 20170521 (per src/tools/pgindent/README),
20201207 (what I happened to have installed before), and 20210402 (the
latest).

Also, what does the difference look like?

- ilmari
-- 
- Twitter seems more influential [than blogs] in the 'gets reported in
  the mainstream press' sense at least.               - Matt McLeod
- That'd be because the content of a tweet is easier to condense down
  to a mainstream media article.                      - Calle Dybedahl

  13. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andrew Dunstan <andrew@dunslane.net> — 2021-05-10T12:29:40Z

    On 5/10/21 7:16 AM, Dagfinn Ilmari Mannsåker wrote:
> Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
>
>> On 07.05.21 20:31, Andrew Dunstan wrote:
>>> On 5/7/21 1:20 PM, Andres Freund wrote:
>>>> On 2021-05-07 11:19:02 -0400, Andrew Dunstan wrote:
>>>>> Here's a patch that adds the README and also adds a Makefile recipe for
>>>>> regenerating Gen_dummy_probes.pl after the sed script is changed. On my
>>>>> system at least the recipe is idempotent.
>>>> Nice! Thanks for this work.
>>> de nada. pushed.
>> This recipe doesn't produce a Gen_dummy_probes.pl that matches exactly
>> the one that is there now.  If this is going to be the preferred method,
>> then we should generate it once so that it matches going forward.
> Which version of perltidy do you have installed?  For me it generates
> identical versions using any of 20170521 (per src/tools/pgindent/README),
> 20201207 (what I happened to have installed before), and 20210402 (the
> latest).
>
> Also, what does the difference look like?
>

Yep:

    andrew@emma:utils $ touch Gen_dummy_probes.sed
    andrew@emma:utils $ touch ../../../src/Makefile.global
    andrew@emma:utils $ make top_srcdir=../../.. Gen_dummy_probes.pl
    perl -ni -e ' print; exit if /^\$0/;' Gen_dummy_probes.pl
    s2p -f Gen_dummy_probes.sed  | sed -e 1,4d -e '/# #/d' -e '$d' >>
    Gen_dummy_probes.pl
    perltidy --profile=../../tools/pgindent/perltidyrc Gen_dummy_probes.pl
    perl -pi -e '!$lb && ( /^\t+#/  || /^# prototypes/ ) && print qq{\n};'\
        -e '$lb = m/^\n/; ' Gen_dummy_probes.pl
    andrew@emma:utils $ git diff
    andrew@emma:utils $ perltidy --version
    This is perltidy, v20170521

cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  14. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-05-10T16:07:48Z

    Andrew Dunstan <andrew@dunslane.net> writes:
> On 5/10/21 7:16 AM, Dagfinn Ilmari Mannsåker wrote:
>> Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
>>> This recipe doesn't produce a Gen_dummy_probes.pl that matches exactly
>>> the one that is there now.  If this is going to be the preferred method,
>>> then we should generate it once so that it matches going forward.

>> Which version of perltidy do you have installed?  For me it generates
>> identical versions using any of 20170521 (per src/tools/pgindent/README),
>> 20201207 (what I happened to have installed before), and 20210402 (the
>> latest).

> Yep:

For me, using App-s2p-1.003 and perltidy v20170521, it works
as long as I start with the previous version of
Gen_dummy_probes.pl in place.  I first tried to test this by
"rm Gen_dummy_probes.pl; make Gen_dummy_probes.pl", and what
I got was a script without all the initial commentary nor
the first line of actual Perl code.

I don't think this is good practice; it implies that any
accidental corruption of the commentary would be carried
forward.  I think we should be extracting the commentary
from Gen_dummy_probes.sed.

			regards, tom lane

  15. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andrew Dunstan <andrew@dunslane.net> — 2021-05-11T14:32:55Z

    On 5/10/21 12:07 PM, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>> On 5/10/21 7:16 AM, Dagfinn Ilmari Mannsåker wrote:
>>> Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
>>>> This recipe doesn't produce a Gen_dummy_probes.pl that matches exactly
>>>> the one that is there now.  If this is going to be the preferred method,
>>>> then we should generate it once so that it matches going forward.
>>> Which version of perltidy do you have installed?  For me it generates
>>> identical versions using any of 20170521 (per src/tools/pgindent/README),
>>> 20201207 (what I happened to have installed before), and 20210402 (the
>>> latest).
>> Yep:
> For me, using App-s2p-1.003 and perltidy v20170521, it works
> as long as I start with the previous version of
> Gen_dummy_probes.pl in place.  I first tried to test this by
> "rm Gen_dummy_probes.pl; make Gen_dummy_probes.pl", and what
> I got was a script without all the initial commentary nor
> the first line of actual Perl code.
>
> I don't think this is good practice; it implies that any
> accidental corruption of the commentary would be carried
> forward.  I think we should be extracting the commentary
> from Gen_dummy_probes.sed.
>
> 			


I don't know how likely accidental corruption is, but OK, let's not make
the next generation dependent on the current generation of the file. The
simplest way around that seems to me to cache the perl prolog, as in the
attached patch Is that more to your liking? I also adjusted it so we
pick up the first line of code from s2p rather than from the prolog,
which is now just comments and the #! line.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  16. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-05-11T14:52:22Z

    Andrew Dunstan <andrew@dunslane.net> writes:
> On 5/10/21 12:07 PM, Tom Lane wrote:
>> I don't think this is good practice; it implies that any
>> accidental corruption of the commentary would be carried
>> forward.  I think we should be extracting the commentary
>> from Gen_dummy_probes.sed.

> I don't know how likely accidental corruption is, but OK, let's not make
> the next generation dependent on the current generation of the file. The
> simplest way around that seems to me to cache the perl prolog, as in the
> attached patch Is that more to your liking? I also adjusted it so we
> pick up the first line of code from s2p rather than from the prolog,
> which is now just comments and the #! line.

Works for me.  One other thought --- do we care whether this works
in a VPATH build, and if so does it?  The $< and $@ references should
be OK, but I'm betting you need $(srcdir)/Gen_dummy_probes.pl.prolog
or the like.

			regards, tom lane

  17. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andrew Dunstan <andrew@dunslane.net> — 2021-05-11T15:44:22Z

    On 5/11/21 10:52 AM, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>> On 5/10/21 12:07 PM, Tom Lane wrote:
>>> I don't think this is good practice; it implies that any
>>> accidental corruption of the commentary would be carried
>>> forward.  I think we should be extracting the commentary
>>> from Gen_dummy_probes.sed.
>> I don't know how likely accidental corruption is, but OK, let's not make
>> the next generation dependent on the current generation of the file. The
>> simplest way around that seems to me to cache the perl prolog, as in the
>> attached patch Is that more to your liking? I also adjusted it so we
>> pick up the first line of code from s2p rather than from the prolog,
>> which is now just comments and the #! line.
> Works for me.  One other thought --- do we care whether this works
> in a VPATH build, and if so does it?  The $< and $@ references should
> be OK, but I'm betting you need $(srcdir)/Gen_dummy_probes.pl.prolog
> or the like.
>
> 			



Why would we? It's only used in Windows builds, and there's no VPATH
there (sadly). In fact, building the file isn't part of any standard
build procedure. I think this is probably in the same boat as the SSL
certs we make in src/test/ssl - I don't think those recipes are meant
for use in VPATH builds either.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  18. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andres Freund <andres@anarazel.de> — 2021-05-11T17:21:15Z

    Hi,

On 2021-05-11 10:52:22 -0400, Tom Lane wrote:
> Works for me.  One other thought --- do we care whether this works
> in a VPATH build, and if so does it?  The $< and $@ references should
> be OK, but I'm betting you need $(srcdir)/Gen_dummy_probes.pl.prolog
> or the like.

It doesn't work in a VPATH build right now, FWIW. $@, $< will point to a
local file in the build directory, right now. And the path to perltidyrc
doesn't work either. It seems to work after the following modifications

diff --git i/src/backend/utils/Makefile w/src/backend/utils/Makefile
index bcf9dd41adf..ca733d12dce 100644
--- i/src/backend/utils/Makefile
+++ w/src/backend/utils/Makefile
@@ -92,10 +92,10 @@ $(top_builddir)/src/include/utils/probes.h: probes.h
 # Nothing depends on it, so it will never be called unless explicitly requested
 # The last two lines of the recipe format the script according to  our
 # standard and put back some blank lines for improved readability.
-Gen_dummy_probes.pl: Gen_dummy_probes.sed
+$(top_srcdir)/src/backend/utils/Gen_dummy_probes.pl: $(top_srcdir)/src/backend/utils/Gen_dummy_probes.sed
     perl -ni -e ' print; exit if /^\$$0/;' $@
     s2p -f $<  | sed -e 1,4d -e '/# #/d' -e '$$d' >> $@
-    perltidy --profile=../../tools/pgindent/perltidyrc $@
+    perltidy --profile=$(top_srcdir)/src/tools/pgindent/perltidyrc $@
     perl -pi -e '!$$lb && ( /^\t+#/  || /^# prototypes/ ) && print qq{\n};'\
         -e '$$lb = m/^\n/; ' $@
 
diff --git i/src/test/regress/parallel_schedule w/src/test/regress/parallel_schedule


Greetings,

Andres Freund

  19. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andres Freund <andres@anarazel.de> — 2021-05-11T17:22:11Z

    Hi,

On 2021-05-11 11:44:22 -0400, Andrew Dunstan wrote:
> Why would we? It's only used in Windows builds, and there's no VPATH
> there (sadly).

Is that really relevant? We'll need to update the file on any platform
when modifying the .sed, not just in windows.

Greetings,

Andres Freund

  20. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andrew Dunstan <andrew@dunslane.net> — 2021-05-11T18:30:10Z

    On 5/11/21 1:21 PM, Andres Freund wrote:
> Hi,
>
> On 2021-05-11 10:52:22 -0400, Tom Lane wrote:
>> Works for me.  One other thought --- do we care whether this works
>> in a VPATH build, and if so does it?  The $< and $@ references should
>> be OK, but I'm betting you need $(srcdir)/Gen_dummy_probes.pl.prolog
>> or the like.
> It doesn't work in a VPATH build right now, FWIW. $@, $< will point to a
> local file in the build directory, right now. And the path to perltidyrc
> doesn't work either. It seems to work after the following modifications
>
> diff --git i/src/backend/utils/Makefile w/src/backend/utils/Makefile
> index bcf9dd41adf..ca733d12dce 100644
> --- i/src/backend/utils/Makefile
> +++ w/src/backend/utils/Makefile
> @@ -92,10 +92,10 @@ $(top_builddir)/src/include/utils/probes.h: probes.h
>  # Nothing depends on it, so it will never be called unless explicitly requested
>  # The last two lines of the recipe format the script according to  our
>  # standard and put back some blank lines for improved readability.
> -Gen_dummy_probes.pl: Gen_dummy_probes.sed
> +$(top_srcdir)/src/backend/utils/Gen_dummy_probes.pl: $(top_srcdir)/src/backend/utils/Gen_dummy_probes.sed
>      perl -ni -e ' print; exit if /^\$$0/;' $@
>      s2p -f $<  | sed -e 1,4d -e '/# #/d' -e '$$d' >> $@
> -    perltidy --profile=../../tools/pgindent/perltidyrc $@
> +    perltidy --profile=$(top_srcdir)/src/tools/pgindent/perltidyrc $@
>      perl -pi -e '!$$lb && ( /^\t+#/  || /^# prototypes/ ) && print qq{\n};'\
>          -e '$$lb = m/^\n/; ' $@
>  


Yeah, but this will create the perl file in the vpath directory where it
won't ever be used anyway. You really want this back in the source
directory where you can check it in etc.

I came up with this:


Gen_dummy_probes.pl: $(top_srcdir)/$(subdir)/Gen_dummy_probes.sed $(top_srcdir)/$(subdir)/Gen_dummy_probes.pl.prolog
    cp $(top_srcdir)/$(subdir)/Gen_dummy_probes.pl.prolog $(top_srcdir)/$(subdir)/$@
    s2p -f $<  | sed -e 1,3d -e '/# #/ d' -e '$$d' >> $(top_srcdir)/$(subdir)/$@
    perltidy --profile=$(top_srcdir)/$(subdir)/../../tools/pgindent/perltidyrc $(top_srcdir)/$(subdir)/$@
    perl -pi -e '!$$lb && ( /^\t+#/  || /^# prototypes/ ) && print qq{\n};'\
        -e '$$lb = m/^\n/; ' $(top_srcdir)/$(subdir)/$@


I'm not aware of any other case where we generate an in-tree file from a
vpath, which is why it feels strange.


cheers


andrew

  21. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andrew Dunstan <andrew@dunslane.net> — 2021-05-11T19:46:40Z

    On 5/11/21 2:30 PM, Andrew Dunstan wrote:
> On 5/11/21 1:21 PM, Andres Freund wrote:
>> Hi,
>>
>> On 2021-05-11 10:52:22 -0400, Tom Lane wrote:
>>> Works for me.  One other thought --- do we care whether this works
>>> in a VPATH build, and if so does it?  The $< and $@ references should
>>> be OK, but I'm betting you need $(srcdir)/Gen_dummy_probes.pl.prolog
>>> or the like.
>> It doesn't work in a VPATH build right now, FWIW. $@, $< will point to a
>> local file in the build directory, right now. And the path to perltidyrc
>> doesn't work either. It seems to work after the following modifications
>>
>> diff --git i/src/backend/utils/Makefile w/src/backend/utils/Makefile
>> index bcf9dd41adf..ca733d12dce 100644
>> --- i/src/backend/utils/Makefile
>> +++ w/src/backend/utils/Makefile
>> @@ -92,10 +92,10 @@ $(top_builddir)/src/include/utils/probes.h: probes.h
>>  # Nothing depends on it, so it will never be called unless explicitly requested
>>  # The last two lines of the recipe format the script according to  our
>>  # standard and put back some blank lines for improved readability.
>> -Gen_dummy_probes.pl: Gen_dummy_probes.sed
>> +$(top_srcdir)/src/backend/utils/Gen_dummy_probes.pl: $(top_srcdir)/src/backend/utils/Gen_dummy_probes.sed
>>      perl -ni -e ' print; exit if /^\$$0/;' $@
>>      s2p -f $<  | sed -e 1,4d -e '/# #/d' -e '$$d' >> $@
>> -    perltidy --profile=../../tools/pgindent/perltidyrc $@
>> +    perltidy --profile=$(top_srcdir)/src/tools/pgindent/perltidyrc $@
>>      perl -pi -e '!$$lb && ( /^\t+#/  || /^# prototypes/ ) && print qq{\n};'\
>>          -e '$$lb = m/^\n/; ' $@
>>  
>
> Yeah, but this will create the perl file in the vpath directory where it
> won't ever be used anyway. You really want this back in the source
> directory where you can check it in etc.
>
> I came up with this:
>
>
> Gen_dummy_probes.pl: $(top_srcdir)/$(subdir)/Gen_dummy_probes.sed $(top_srcdir)/$(subdir)/Gen_dummy_probes.pl.prolog
>     cp $(top_srcdir)/$(subdir)/Gen_dummy_probes.pl.prolog $(top_srcdir)/$(subdir)/$@
>     s2p -f $<  | sed -e 1,3d -e '/# #/ d' -e '$$d' >> $(top_srcdir)/$(subdir)/$@
>     perltidy --profile=$(top_srcdir)/$(subdir)/../../tools/pgindent/perltidyrc $(top_srcdir)/$(subdir)/$@
>     perl -pi -e '!$$lb && ( /^\t+#/  || /^# prototypes/ ) && print qq{\n};'\
>         -e '$$lb = m/^\n/; ' $(top_srcdir)/$(subdir)/$@
>
>
> I'm not aware of any other case where we generate an in-tree file from a
> vpath, which is why it feels strange.



Simplified version:


Gen_dummy_probes.pl: $(srcdir)/Gen_dummy_probes.sed $(srcdir)/Gen_dummy_probes.pl.prolog
    cp $(srcdir)/Gen_dummy_probes.pl.prolog $(srcdir)/$@
    s2p -f $<  | sed -e 1,3d -e '/# #/ d' -e '$$d' >> $(srcdir)/$@
    perltidy --profile=$(srcdir)/../../tools/pgindent/perltidyrc $(srcdir)/$@
    perl -pi -e '!$$lb && ( /^\t+#/  || /^# prototypes/ ) && print qq{\n};'\
        -e '$$lb = m/^\n/; ' $(srcdir)/$@

cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  22. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andres Freund <andres@anarazel.de> — 2021-05-11T19:47:45Z

    Hi,

On 2021-05-11 14:30:10 -0400, Andrew Dunstan wrote:
> On 5/11/21 1:21 PM, Andres Freund wrote:
> > It doesn't work in a VPATH build right now, FWIW. $@, $< will point to a
> > local file in the build directory, right now. And the path to perltidyrc
> > doesn't work either. It seems to work after the following modifications
> >
> > diff --git i/src/backend/utils/Makefile w/src/backend/utils/Makefile
> > index bcf9dd41adf..ca733d12dce 100644
> > --- i/src/backend/utils/Makefile
> > +++ w/src/backend/utils/Makefile
> > @@ -92,10 +92,10 @@ $(top_builddir)/src/include/utils/probes.h: probes.h
> >  # Nothing depends on it, so it will never be called unless explicitly requested
> >  # The last two lines of the recipe format the script according to  our
> >  # standard and put back some blank lines for improved readability.
> > -Gen_dummy_probes.pl: Gen_dummy_probes.sed
> > +$(top_srcdir)/src/backend/utils/Gen_dummy_probes.pl: $(top_srcdir)/src/backend/utils/Gen_dummy_probes.sed
> >      perl -ni -e ' print; exit if /^\$$0/;' $@
> >      s2p -f $<  | sed -e 1,4d -e '/# #/d' -e '$$d' >> $@
> > -    perltidy --profile=../../tools/pgindent/perltidyrc $@
> > +    perltidy --profile=$(top_srcdir)/src/tools/pgindent/perltidyrc $@
> >      perl -pi -e '!$$lb && ( /^\t+#/  || /^# prototypes/ ) && print qq{\n};'\
> >          -e '$$lb = m/^\n/; ' $@
> >  
> 
> 
> Yeah, but this will create the perl file in the vpath directory where it
> won't ever be used anyway. You really want this back in the source
> directory where you can check it in etc.

Hm, why would it? Both $< and $@ will be in the source tree, as
formulated above?

andres@awork3:~/src/postgresql$ git diff src/backend/utils/Gen_dummy_probes.pl
andres@awork3:~/src/postgresql$ echo foo >> src/backend/utils/Gen_dummy_probes.pl
andres@awork3:~/src/postgresql$ git diff src/backend/utils/Gen_dummy_probes.pl
diff --git i/src/backend/utils/Gen_dummy_probes.pl w/src/backend/utils/Gen_dummy_probes.pl
index 4852103daf4..1b724bf1038 100644
--- i/src/backend/utils/Gen_dummy_probes.pl
+++ w/src/backend/utils/Gen_dummy_probes.pl
@@ -257,3 +257,4 @@ sub printQ()
     }
     undef(@Q);
 }
+foo
andres@awork3:~/src/postgresql$ touch src/backend/utils/Gen_dummy_probes.sed && make -C /home/andres/build/postgres/dev-assert/vpath/src/backend/utils Gen_dummy_probes.pl
perl -ni -e ' print; exit if /^\$0/;' /home/andres/src/postgresql/src/backend/utils/Gen_dummy_probes.pl
s2p -f /home/andres/src/postgresql/src/backend/utils/Gen_dummy_probes.sed  | sed -e 1,4d -e '/# #/d' -e '$d' >> /home/andres/src/postgresql/src/backend/utils/Gen_dummy_probes.pl
perltidy --profile=/home/andres/src/postgresql/src/tools/pgindent/perltidyrc /home/andres/src/postgresql/src/backend/utils/Gen_dummy_probes.pl
perl -pi -e '!$lb && ( /^\t+#/  || /^# prototypes/ ) && print qq{\n};'\
	-e '$lb = m/^\n/; ' /home/andres/src/postgresql/src/backend/utils/Gen_dummy_probes.pl
andres@awork3:~/src/postgresql$ git diff src/backend/utils/Gen_dummy_probes.pl
andres@awork3:~/src/postgresql$ ls /home/andres/build/postgres/dev-assert/vpath/src/backend/utils/Gen_dummy_probes*
/home/andres/build/postgres/dev-assert/vpath/src/backend/utils/Gen_dummy_probes.pl.tdy

So only a temp file ends up in the build dir?


> I came up with this:
> 
> 
> Gen_dummy_probes.pl: $(top_srcdir)/$(subdir)/Gen_dummy_probes.sed $(top_srcdir)/$(subdir)/Gen_dummy_probes.pl.prolog
>     cp $(top_srcdir)/$(subdir)/Gen_dummy_probes.pl.prolog $(top_srcdir)/$(subdir)/$@
>     s2p -f $<  | sed -e 1,3d -e '/# #/ d' -e '$$d' >> $(top_srcdir)/$(subdir)/$@
>     perltidy --profile=$(top_srcdir)/$(subdir)/../../tools/pgindent/perltidyrc $(top_srcdir)/$(subdir)/$@
>     perl -pi -e '!$$lb && ( /^\t+#/  || /^# prototypes/ ) && print qq{\n};'\
>         -e '$$lb = m/^\n/; ' $(top_srcdir)/$(subdir)/$@
> 
> 
> I'm not aware of any other case where we generate an in-tree file from a
> vpath, which is why it feels strange.

Yea, it is a bit odd, agreed. We don't have many generated sources
inside the git repo (vs in the tarball). The most prominent one is
configure, obviously...

Greetings,

Andres Freund

  23. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2021-05-11T19:53:06Z

    On 2021-May-11, Andrew Dunstan wrote:

> Yeah, but this will create the perl file in the vpath directory where it
> won't ever be used anyway. You really want this back in the source
> directory where you can check it in etc.

Hmm ... no, it doesn't.  As far as I can see it works as intended.  The
path to the prolog file is broken, yes, but that's it.  However, maybe
I'm doing something wrong but if you define the rule like that, then it
no longer works in normal (non-vpath) mode.  This works:


-- 
Álvaro Herrera                            39°49'30"S 73°17'W

  24. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-05-11T20:01:28Z

    Andres Freund <andres@anarazel.de> writes:
> On 2021-05-11 14:30:10 -0400, Andrew Dunstan wrote:
>> I'm not aware of any other case where we generate an in-tree file from a
>> vpath, which is why it feels strange.

> Yea, it is a bit odd, agreed. We don't have many generated sources
> inside the git repo (vs in the tarball). The most prominent one is
> configure, obviously...

I think this is overly cute.  As a counterexample, the rules to regenerate
gram.c and similar files don't bend over backwards like that to force the
output to be in the srcdir.

I haven't dug in the gmake manual to be sure, but I think that in a VPATH
build, $@ will refer to the file in the srcdir if the file exists there
but is out-of-date.  So if you go with the straightforward use of $< and
$@, I believe it will in fact work.  The only way to make it fail under
VPATH would be to do
	rm path/to/srcdir/Gen_dummy_probes.pl; make Gen_dummy_probes.pl
which I think is sufficiently unlikely to not be a problem.  In fact,
one could argue that building Gen_dummy_probes.pl in the VPATH dir
is exactly what the user is trying to make happen if she does this.

In short: don't be cuter than the longstanding bison/flex rules are.

			regards, tom lane

  25. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andrew Dunstan <andrew@dunslane.net> — 2021-05-11T21:43:29Z

    On 5/11/21 4:01 PM, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
>> On 2021-05-11 14:30:10 -0400, Andrew Dunstan wrote:
>>> I'm not aware of any other case where we generate an in-tree file from a
>>> vpath, which is why it feels strange.
>> Yea, it is a bit odd, agreed. We don't have many generated sources
>> inside the git repo (vs in the tarball). The most prominent one is
>> configure, obviously...
> I think this is overly cute.  As a counterexample, the rules to regenerate
> gram.c and similar files don't bend over backwards like that to force the
> output to be in the srcdir.
>
> I haven't dug in the gmake manual to be sure, but I think that in a VPATH
> build, $@ will refer to the file in the srcdir if the file exists there
> but is out-of-date.  So if you go with the straightforward use of $< and
> $@, I believe it will in fact work.  The only way to make it fail under
> VPATH would be to do
> 	rm path/to/srcdir/Gen_dummy_probes.pl; make Gen_dummy_probes.pl
> which I think is sufficiently unlikely to not be a problem.  In fact,
> one could argue that building Gen_dummy_probes.pl in the VPATH dir
> is exactly what the user is trying to make happen if she does this.
>
> In short: don't be cuter than the longstanding bison/flex rules are.
>
> 			

What will she do with it? gram.c generated in a vpath build is 100%
usable where it's generated. Also. it's not a file we keep in the git repo.

Not gonna fight, there's been way too much energy spent on this. I'll
just do what Alvaro suggested. But I won't be surprised if some future
commit is missing the perl update.


cheers


andrew




--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  26. Re: Why do we have perl and sed versions of Gen_dummy_probes?

    Andrew Dunstan <andrew@dunslane.net> — 2021-05-11T21:54:33Z

    On 5/11/21 5:43 PM, Andrew Dunstan wrote:
> On 5/11/21 4:01 PM, Tom Lane wrote:
>> Andres Freund <andres@anarazel.de> writes:
>>> On 2021-05-11 14:30:10 -0400, Andrew Dunstan wrote:
>>>> I'm not aware of any other case where we generate an in-tree file from a
>>>> vpath, which is why it feels strange.
>>> Yea, it is a bit odd, agreed. We don't have many generated sources
>>> inside the git repo (vs in the tarball). The most prominent one is
>>> configure, obviously...
>> I think this is overly cute.  As a counterexample, the rules to regenerate
>> gram.c and similar files don't bend over backwards like that to force the
>> output to be in the srcdir.
>>
>> I haven't dug in the gmake manual to be sure, but I think that in a VPATH
>> build, $@ will refer to the file in the srcdir if the file exists there
>> but is out-of-date.  So if you go with the straightforward use of $< and
>> $@, I believe it will in fact work.  The only way to make it fail under
>> VPATH would be to do
>> 	rm path/to/srcdir/Gen_dummy_probes.pl; make Gen_dummy_probes.pl
>> which I think is sufficiently unlikely to not be a problem.  In fact,
>> one could argue that building Gen_dummy_probes.pl in the VPATH dir
>> is exactly what the user is trying to make happen if she does this.
>>
>> In short: don't be cuter than the longstanding bison/flex rules are.
>>
>> 			
> What will she do with it? gram.c generated in a vpath build is 100%
> usable where it's generated. Also. it's not a file we keep in the git repo.
>
> Not gonna fight, there's been way too much energy spent on this. I'll
> just do what Alvaro suggested. But I won't be surprised if some future
> commit is missing the perl update.
>
>


Belay that. His patch does what I tried to do but does it right. I'll
figure it out.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com