Horton

Thread

Commits

  1. meson: Stop using deprecated way getting path of files

  1. meson: Stop using deprecated way getting path of files

    Andres Freund <andres@anarazel.de> — 2023-11-29T18:50:53Z

    Hi,

The just released meson 1.3 strongly deprecated a hack we were using, emitting
a noisy warning (the hack basically depended on an implementation detail to
work). Turns out there has been a better way available for a while, I just
hadn't found it. 1.4 added a more convenient approach, but we can't rely on
that.

Everything continues to work, but the warning is annoying.

The warning:

Message: checking for file conflicts between source and build directory
../home/andres/src/postgresql/meson.build:2972: DEPRECATION: Project uses feature that was always broken, and is now deprecated since '1.3.0': str.format: Value other than strings, integers, bools, options, dictionaries and lists thereof..
[...]
WARNING: Broken features used:
 * 1.3.0: {'str.format: Value other than strings, integers, bools, options, dictionaries and lists thereof.'}

I plan to apply this soon, unless I hear some opposition / better ideas / ....

Greetings,

Andres Freund

  2. Re: meson: Stop using deprecated way getting path of files

    Tristan Partin <tristan@neon.tech> — 2023-11-29T19:11:23Z

    This looks good to me. What is our limiting factor on bumping the 
minimum Meson version?

While we are on the topic of Meson, it would be great if you could take 
a look at this thread[0], where I am trying to compile Postgres with 
-fsanitize=address,undefined (-Db_sanitize=address,undefined).

[0]: https://www.postgresql.org/message-id/CWTM35CAUKRT.1733OSMXUZW7%40neon.tech

-- 
Tristan Partin
Neon (https://neon.tech)

  3. Re: meson: Stop using deprecated way getting path of files

    Andres Freund <andres@anarazel.de> — 2023-11-29T19:42:24Z

    Hi,

On 2023-11-29 13:11:23 -0600, Tristan Partin wrote:
> This looks good to me.

Cool.


> What is our limiting factor on bumping the minimum Meson version?

Old distro versions, particularly ones where the distro just has an older
python. It's one thing to require installing meson but quite another to also
require building python. There's some other ongoing discussion about
establishing the minimum baseline in a somewhat more, uh, formalized way:
https://postgr.es/m/CA%2BhUKGLhNs5geZaVNj2EJ79Dx9W8fyWUU3HxcpZy55sMGcY%3DiA%40mail.gmail.com

> While we are on the topic of Meson, it would be great if you could take a
> look at this thread[0], where I am trying to compile Postgres with
> -fsanitize=address,undefined (-Db_sanitize=address,undefined).

Done. Not sure it helps you much though :)

Greetings,

Andres Freund

  4. Re: meson: Stop using deprecated way getting path of files

    Tristan Partin <tristan@neon.tech> — 2023-11-30T21:00:22Z

    On Wed Nov 29, 2023 at 1:42 PM CST, Andres Freund wrote:
> Hi,
>
> On 2023-11-29 13:11:23 -0600, Tristan Partin wrote:
> > What is our limiting factor on bumping the minimum Meson version?
>
> Old distro versions, particularly ones where the distro just has an older
> python. It's one thing to require installing meson but quite another to also
> require building python. There's some other ongoing discussion about
> establishing the minimum baseline in a somewhat more, uh, formalized way:
> https://postgr.es/m/CA%2BhUKGLhNs5geZaVNj2EJ79Dx9W8fyWUU3HxcpZy55sMGcY%3DiA%40mail.gmail.com

I'll take a look there. According to Meson, the following versions had 
Python version bumps:

0.61.5: 3.6
0.56.2: 3.5
0.45.1: 3.4

Taking a look at pkgs.org, Debian 10, Ubuntu 20.04, and Oracle Linux 
7 (a RHEL re-spin), and CentOS 7, all have >= Python 3.6.8. Granted, 
this isn't the whole picture of what Postgres supports from version 16+. 
To put things in perspective, Python 3.6 was released on December 23, 
2016, which is coming up on 7 years. Python 3.6 reached end of life on 
the same date in 2021.

Is there a complete list somewhere that talks about what platforms each 
version of Postgres supports?

-- 
Tristan Partin
Neon (https://neon.tech)

  5. Re: meson: Stop using deprecated way getting path of files

    Andrew Dunstan <andrew@dunslane.net> — 2023-11-30T21:46:06Z

    On 2023-11-30 Th 16:00, Tristan Partin wrote:
> On Wed Nov 29, 2023 at 1:42 PM CST, Andres Freund wrote:
>> Hi,
>>
>> On 2023-11-29 13:11:23 -0600, Tristan Partin wrote:
>> > What is our limiting factor on bumping the minimum Meson version?
>>
>> Old distro versions, particularly ones where the distro just has an 
>> older
>> python. It's one thing to require installing meson but quite another 
>> to also
>> require building python. There's some other ongoing discussion about
>> establishing the minimum baseline in a somewhat more, uh, formalized 
>> way:
>> https://postgr.es/m/CA%2BhUKGLhNs5geZaVNj2EJ79Dx9W8fyWUU3HxcpZy55sMGcY%3DiA%40mail.gmail.com 
>>
>
> I'll take a look there. According to Meson, the following versions had 
> Python version bumps:
>
> 0.61.5: 3.6
> 0.56.2: 3.5
> 0.45.1: 3.4
>
> Taking a look at pkgs.org, Debian 10, Ubuntu 20.04, and Oracle Linux 7 
> (a RHEL re-spin), and CentOS 7, all have >= Python 3.6.8. Granted, 
> this isn't the whole picture of what Postgres supports from version 
> 16+. To put things in perspective, Python 3.6 was released on December 
> 23, 2016, which is coming up on 7 years. Python 3.6 reached end of 
> life on the same date in 2021.
>
> Is there a complete list somewhere that talks about what platforms 
> each version of Postgres supports?


You can look at animals in the buildfarm. For meson only release 16 and 
up matter.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  6. Re: meson: Stop using deprecated way getting path of files

    Andres Freund <andres@anarazel.de> — 2023-12-01T04:06:01Z

    On 2023-11-29 10:50:53 -0800, Andres Freund wrote:
> I plan to apply this soon, unless I hear some opposition / better ideas / ....

Pushed.

  7. Re: meson: Stop using deprecated way getting path of files

    Tristan Partin <tristan@neon.tech> — 2023-12-01T16:18:38Z

    On Thu Nov 30, 2023 at 3:46 PM CST, Andrew Dunstan wrote:
>
> On 2023-11-30 Th 16:00, Tristan Partin wrote:
> > On Wed Nov 29, 2023 at 1:42 PM CST, Andres Freund wrote:
> >> Hi,
> >>
> >> On 2023-11-29 13:11:23 -0600, Tristan Partin wrote:
> >> > What is our limiting factor on bumping the minimum Meson version?
> >>
> >> Old distro versions, particularly ones where the distro just has an 
> >> older
> >> python. It's one thing to require installing meson but quite another 
> >> to also
> >> require building python. There's some other ongoing discussion about
> >> establishing the minimum baseline in a somewhat more, uh, formalized 
> >> way:
> >> https://postgr.es/m/CA%2BhUKGLhNs5geZaVNj2EJ79Dx9W8fyWUU3HxcpZy55sMGcY%3DiA%40mail.gmail.com 
> >>
> >
> > I'll take a look there. According to Meson, the following versions had 
> > Python version bumps:
> >
> > 0.61.5: 3.6
> > 0.56.2: 3.5
> > 0.45.1: 3.4
> >
> > Taking a look at pkgs.org, Debian 10, Ubuntu 20.04, and Oracle Linux 7 
> > (a RHEL re-spin), and CentOS 7, all have >= Python 3.6.8. Granted, 
> > this isn't the whole picture of what Postgres supports from version 
> > 16+. To put things in perspective, Python 3.6 was released on December 
> > 23, 2016, which is coming up on 7 years. Python 3.6 reached end of 
> > life on the same date in 2021.
> >
> > Is there a complete list somewhere that talks about what platforms 
> > each version of Postgres supports?
>
>
> You can look at animals in the buildfarm. For meson only release 16 and 
> up matter.

On the buildfarm page[0], it would be nice if more than just the 
compiler versions were stated. It would be nice to have all 
build/runtime dependencies listed. For instance, it would be interesting 
if there was a json document for each build animal, and perhaps a root 
json document which was an amalgomation of the individual documents. 
Then I could use a tool like jq to query all the information rather 
easily. As-is, I don't know where to search for package versions for 
some of the archaic operating systems in the farm. Perhaps other people 
have had similar problems in the past. Having a full write-up of every 
build machine would also be good for debugging purposes. If I see 
openssl tests suddenly failing on one machine, then I can just check the 
openssl version, and try to reproduce locally.

I know the buildfarm seems to be a volunteer thing, so asking more of 
them seems like a hard ask. Just wanted to throw my thoughts into the 
void.

[0]: https://buildfarm.postgresql.org/cgi-bin/show_members.pl

-- 
Tristan Partin
Neon (https://neon.tech)

  8. Re: meson: Stop using deprecated way getting path of files

    Tom Lane <tgl@sss.pgh.pa.us> — 2023-12-01T18:07:02Z

    "Tristan Partin" <tristan@neon.tech> writes:
> On the buildfarm page[0], it would be nice if more than just the 
> compiler versions were stated. It would be nice to have all 
> build/runtime dependencies listed.

By and large, we've attempted to address such concerns by extending
the configure script to emit info about versions of things it finds.
So you should look into the configure step's log to see what version
of bison, openssl, etc is in use.

> I know the buildfarm seems to be a volunteer thing, so asking more of 
> them seems like a hard ask.

We certainly aren't going to ask owners to maintain such information
manually.  Even if they tried, it'd soon be impossibly out-of-date.
The logging method has the additional advantage that it's accurate
for historical runs as well as whatever the machine has installed
today.

			regards, tom lane

  9. Re: meson: Stop using deprecated way getting path of files

    Tristan Partin <tristan@neon.tech> — 2023-12-01T18:16:38Z

    On Fri Dec 1, 2023 at 12:07 PM CST, Tom Lane wrote:
> "Tristan Partin" <tristan@neon.tech> writes:
> > On the buildfarm page[0], it would be nice if more than just the 
> > compiler versions were stated. It would be nice to have all 
> > build/runtime dependencies listed.
>
> By and large, we've attempted to address such concerns by extending
> the configure script to emit info about versions of things it finds.
> So you should look into the configure step's log to see what version
> of bison, openssl, etc is in use.

Good point. For some reason that slipped my mind. Off into the weeds 
I go...

-- 
Tristan Partin
Neon (https://neon.tech)

  10. Re: meson: Stop using deprecated way getting path of files

    Tristan Partin <tristan@neon.tech> — 2023-12-04T19:55:17Z

    On Fri Dec 1, 2023 at 12:16 PM CST, Tristan Partin wrote:
> On Fri Dec 1, 2023 at 12:07 PM CST, Tom Lane wrote:
> > "Tristan Partin" <tristan@neon.tech> writes:
> > > On the buildfarm page[0], it would be nice if more than just the 
> > > compiler versions were stated. It would be nice to have all 
> > > build/runtime dependencies listed.
> >
> > By and large, we've attempted to address such concerns by extending
> > the configure script to emit info about versions of things it finds.
> > So you should look into the configure step's log to see what version
> > of bison, openssl, etc is in use.
>
> Good point. For some reason that slipped my mind. Off into the weeds 
> I go...

Ok, so what I found is that we still have build farm animals using 
Python 3.4, specifically the AIX machines. There was also at least one 
Python 3.5 user too. Note that this was a manual check.

I think I'll probably work on a tool for querying information out of the 
build farm tonight to make tasks like this much more automated.

-- 
Tristan Partin
Neon (https://neon.tech)

  11. Re: meson: Stop using deprecated way getting path of files

    Tom Lane <tgl@sss.pgh.pa.us> — 2023-12-04T20:10:25Z

    "Tristan Partin" <tristan@neon.tech> writes:
> On Fri Dec 1, 2023 at 12:16 PM CST, Tristan Partin wrote:
>>> Ok, so what I found is that we still have build farm animals using 
>>> Python 3.4, specifically the AIX machines. There was also at least one 
>>> Python 3.5 user too. Note that this was a manual check.

> I think I'll probably work on a tool for querying information out of the 
> build farm tonight to make tasks like this much more automated.

Not sure what you were using, but are you aware that SQL access to the
buildfarm database is available to project members?  My own stock
approach to checking on this sort of thing is like

select * from
(select sysname, snapshot, unnest(string_to_array(log_text, E'\n')) as l
    from build_status_log join snapshots using (sysname, snapshot)
    where log_stage = 'configure.log') ss
where l like 'checking for builtin %'

			regards, tom lane

  12. Re: meson: Stop using deprecated way getting path of files

    Tristan Partin <tristan@neon.tech> — 2023-12-04T20:26:42Z

    On Mon Dec 4, 2023 at 2:10 PM CST, Tom Lane wrote:
> "Tristan Partin" <tristan@neon.tech> writes:
> > On Fri Dec 1, 2023 at 12:16 PM CST, Tristan Partin wrote:
> >>> Ok, so what I found is that we still have build farm animals using 
> >>> Python 3.4, specifically the AIX machines. There was also at least one 
> >>> Python 3.5 user too. Note that this was a manual check.
>
> > I think I'll probably work on a tool for querying information out of the 
> > build farm tonight to make tasks like this much more automated.
>
> Not sure what you were using, but are you aware that SQL access to the
> buildfarm database is available to project members?  My own stock
> approach to checking on this sort of thing is like
>
> select * from
> (select sysname, snapshot, unnest(string_to_array(log_text, E'\n')) as l
>     from build_status_log join snapshots using (sysname, snapshot)
>     where log_stage = 'configure.log') ss
> where l like 'checking for builtin %'

This looks useful. I had no idea about this. Can you send me any 
resources for setting this up? My idea was just to do some web scraping.

-- 
Tristan Partin
Neon (https://neon.tech)

  13. Re: meson: Stop using deprecated way getting path of files

    John Naylor <johncnaylorls@gmail.com> — 2023-12-18T06:43:09Z

    On Tue, Dec 5, 2023 at 3:27 AM Tristan Partin <tristan@neon.tech> wrote:
>
> On Mon Dec 4, 2023 at 2:10 PM CST, Tom Lane wrote:
> > Not sure what you were using, but are you aware that SQL access to the
> > buildfarm database is available to project members?  My own stock
> > approach to checking on this sort of thing is like
> >
> > select * from
> > (select sysname, snapshot, unnest(string_to_array(log_text, E'\n')) as l
> >     from build_status_log join snapshots using (sysname, snapshot)
> >     where log_stage = 'configure.log') ss
> > where l like 'checking for builtin %'
>
> This looks useful. I had no idea about this. Can you send me any
> resources for setting this up? My idea was just to do some web scraping.

+1 -- I was vaguely aware of this, but can't find any mention of
specifics in the buildfarm how-to, or other places I thought to look.

  14. Re: meson: Stop using deprecated way getting path of files

    Tristan Partin <tristan@neon.tech> — 2023-12-19T16:08:10Z

    On Mon Dec 18, 2023 at 12:43 AM CST, John Naylor wrote:
> On Tue, Dec 5, 2023 at 3:27 AM Tristan Partin <tristan@neon.tech> wrote:
> >
> > On Mon Dec 4, 2023 at 2:10 PM CST, Tom Lane wrote:
> > > Not sure what you were using, but are you aware that SQL access to the
> > > buildfarm database is available to project members?  My own stock
> > > approach to checking on this sort of thing is like
> > >
> > > select * from
> > > (select sysname, snapshot, unnest(string_to_array(log_text, E'\n')) as l
> > >     from build_status_log join snapshots using (sysname, snapshot)
> > >     where log_stage = 'configure.log') ss
> > > where l like 'checking for builtin %'
> >
> > This looks useful. I had no idea about this. Can you send me any
> > resources for setting this up? My idea was just to do some web scraping.
>
> +1 -- I was vaguely aware of this, but can't find any mention of
> specifics in the buildfarm how-to, or other places I thought to look.

From my off-list conversations with Andrew, database access to the 
buildfarm is for trusted contributors. I do not meet current criteria. 
I've thought about building a web-scraper to get at some of this 
information for non-trusted contributors. If that interests you, let me 
know, and maybe I can build it out over the holiday. Or maybe you meet 
the criteria! :)

-- 
Tristan Partin
Neon (https://neon.tech)