Horton
Menu

Thread

  1. testing plpython3u on 9.0beta2

    Chris <rfusca@gmail.com> — 2010-06-23T02:41:13Z

    I received two errors (described below) in installing 9.0beta2 on Kubuntu
10.04 , RhodiumToad on IRC recommended I post them here.

I did not have a 2.x or 3.x python dev installed, but I was really only
interested in python3 via plython3u.
So...
sudo apt-get install python3-all-dev

Configure works fine...
"./configure --with-pgport=5433 --with-python --with-ossp-uuid --with-libxml
--with-libxslt --with-perl"
"make" fails while trying to build plpython, Cannot find python.h i
believe....(I sadly did not save the text of the error and have since
continued onward.  But it was not finding a file while building plpython.  I
believe I would have to uninstall a few things to reproduce.)
So I tried installing the 2.x dev.  After running "sudo apt-get install
python-all-dev" then "make;make check;sudo make install", it all worke fine.

However,  when issuing a "createlang plpython3u template1", I get
"createlang: language installation failed: ERROR:  could not access file
"$libdir/plpython3": No such file or directory".  But if I "createlang
plpython2u template1" first, then it will allow "createlang plpython3u
template1".

Am I doing something incorrect, or...?

(As an aside, functions created with LANGUAGE plpython3u appear to work.)


-- 
Chris Spotts
rfusca@gmail.com

  2. Re: testing plpython3u on 9.0beta2

    Robert Haas <robertmhaas@gmail.com> — 2010-06-23T11:17:12Z

    On Tue, Jun 22, 2010 at 10:41 PM, Chris <rfusca@gmail.com> wrote:
> I received two errors (described below) in installing 9.0beta2 on Kubuntu
> 10.04 , RhodiumToad on IRC recommended I post them here.
> I did not have a 2.x or 3.x python dev installed, but I was really only
> interested in python3 via plython3u.
> So...
> sudo apt-get install python3-all-dev
> Configure works fine...
> "./configure --with-pgport=5433 --with-python --with-ossp-uuid --with-libxml
> --with-libxslt --with-perl"
> "make" fails while trying to build plpython, Cannot find python.h i
> believe....(I sadly did not save the text of the error and have since
> continued onward.  But it was not finding a file while building plpython.  I
> believe I would have to uninstall a few things to reproduce.)
> So I tried installing the 2.x dev.  After running "sudo apt-get install
> python-all-dev" then "make;make check;sudo make install", it all worke fine.
> However,  when issuing a "createlang plpython3u template1", I get
> "createlang: language installation failed: ERROR:  could not access file
> "$libdir/plpython3": No such file or directory".  But if I "createlang
> plpython2u template1" first, then it will allow "createlang plpython3u
> template1".
> Am I doing something incorrect, or...?
> (As an aside, functions created with LANGUAGE plpython3u appear to work.)

I can reproduce this, here.  The problem seems to be that plpython
only build either plpython2.so or plython3.so, but both languages
expect a call handler called plython_call_handler.  So once we load
the shared library for one language, the other language just grabs the
same call handler.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

  3. Re: testing plpython3u on 9.0beta2

    Tom Lane <tgl@sss.pgh.pa.us> — 2010-06-23T14:30:54Z

    Robert Haas <robertmhaas@gmail.com> writes:
> I can reproduce this, here.  The problem seems to be that plpython
> only build either plpython2.so or plython3.so, but both languages
> expect a call handler called plython_call_handler.  So once we load
> the shared library for one language, the other language just grabs the
> same call handler.

We had better fix that --- I can definitely foresee installations
wanting to use both plpython2 and plpython3.  This'd require a change in
the default contents of pg_pltemplate, though.  Does it seem important
enough to bump catversion for?

			regards, tom lane

  4. Re: testing plpython3u on 9.0beta2

    Robert Haas <robertmhaas@gmail.com> — 2010-06-23T14:49:02Z

    On Wed, Jun 23, 2010 at 10:30 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> I can reproduce this, here.  The problem seems to be that plpython
>> only build either plpython2.so or plython3.so, but both languages
>> expect a call handler called plython_call_handler.  So once we load
>> the shared library for one language, the other language just grabs the
>> same call handler.
>
> We had better fix that --- I can definitely foresee installations
> wanting to use both plpython2 and plpython3.  This'd require a change in
> the default contents of pg_pltemplate, though.  Does it seem important
> enough to bump catversion for?

Yeah, I think so.  As for using both plpython2 and plpython3, it looks
to me like right now you can only use one or the other.  I think if
you somehow manage to install both, you're really just getting the
same one twice (I have not tested this, however).

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

  5. Re: testing plpython3u on 9.0beta2

    Robert Haas <robertmhaas@gmail.com> — 2010-06-24T16:56:40Z

    On Wed, Jun 23, 2010 at 10:49 AM, Robert Haas <robertmhaas@gmail.com> wrote:
> On Wed, Jun 23, 2010 at 10:30 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Robert Haas <robertmhaas@gmail.com> writes:
>>> I can reproduce this, here.  The problem seems to be that plpython
>>> only build either plpython2.so or plython3.so, but both languages
>>> expect a call handler called plython_call_handler.  So once we load
>>> the shared library for one language, the other language just grabs the
>>> same call handler.
>>
>> We had better fix that --- I can definitely foresee installations
>> wanting to use both plpython2 and plpython3.  This'd require a change in
>> the default contents of pg_pltemplate, though.  Does it seem important
>> enough to bump catversion for?
>
> Yeah, I think so.  As for using both plpython2 and plpython3, it looks
> to me like right now you can only use one or the other.  I think if
> you somehow manage to install both, you're really just getting the
> same one twice (I have not tested this, however).

So, what's the right thing to do here?  Should we just fix it so that
creating the second language always fails, or should we try to make it
possible for both of them to coexist (which is probably a lot more
work)?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

  6. Re: testing plpython3u on 9.0beta2

    Peter Eisentraut <peter_e@gmx.net> — 2010-06-25T05:57:47Z

    On ons, 2010-06-23 at 07:17 -0400, Robert Haas wrote:
> I can reproduce this, here.  The problem seems to be that plpython
> only build either plpython2.so or plython3.so, but both languages
> expect a call handler called plython_call_handler.  So once we load
> the shared library for one language, the other language just grabs the
> same call handler.

The problem is apparently that when CREATE LANGUAGE creates a language
from a pg_pltemplate entry, it creates the proname from the tmplhandler
name, and if it finds a fitting proname entry already, it used that one.
So when you create plpython2 first and plpython3 second, the pg_language
entries of the latter point to the pg_proc entries of the former.

If you fix that up manually (create additional pg_proc entries and fix
the pg_language entries to point there), it works better.

  7. Re: testing plpython3u on 9.0beta2

    Tom Lane <tgl@sss.pgh.pa.us> — 2010-06-25T14:17:32Z

    Peter Eisentraut <peter_e@gmx.net> writes:
> The problem is apparently that when CREATE LANGUAGE creates a language
> from a pg_pltemplate entry, it creates the proname from the tmplhandler
> name, and if it finds a fitting proname entry already, it used that one.
> So when you create plpython2 first and plpython3 second, the pg_language
> entries of the latter point to the pg_proc entries of the former.

> If you fix that up manually (create additional pg_proc entries and fix
> the pg_language entries to point there), it works better.

The fix ought to be to change the function nmes used by plpython3 ...

			regards, tom lane

  8. Re: testing plpython3u on 9.0beta2

    Peter Eisentraut <peter_e@gmx.net> — 2010-06-25T18:49:23Z

    On fre, 2010-06-25 at 10:17 -0400, Tom Lane wrote:
> Peter Eisentraut <peter_e@gmx.net> writes:
> > The problem is apparently that when CREATE LANGUAGE creates a language
> > from a pg_pltemplate entry, it creates the proname from the tmplhandler
> > name, and if it finds a fitting proname entry already, it used that one.
> > So when you create plpython2 first and plpython3 second, the pg_language
> > entries of the latter point to the pg_proc entries of the former.
> 
> > If you fix that up manually (create additional pg_proc entries and fix
> > the pg_language entries to point there), it works better.
> 
> The fix ought to be to change the function nmes used by plpython3 ...

Right.  What shall we do about the catversion?

  9. Re: testing plpython3u on 9.0beta2

    Tom Lane <tgl@sss.pgh.pa.us> — 2010-06-25T19:07:16Z

    Peter Eisentraut <peter_e@gmx.net> writes:
> On fre, 2010-06-25 at 10:17 -0400, Tom Lane wrote:
>> The fix ought to be to change the function nmes used by plpython3 ...

> Right.  What shall we do about the catversion?

You could argue it either way.  The number of beta testers with
plpython3 installations is probably very small, so I'm kinda leaning to
just changing the code without a catversion bump.  OTOH, if we want to
encourage testing of pg_upgrade ...

			regards, tom lane

  10. Re: testing plpython3u on 9.0beta2

    Josh Berkus <josh@agliodbs.com> — 2010-06-25T19:09:23Z

    > You could argue it either way.  The number of beta testers with
> plpython3 installations is probably very small, so I'm kinda leaning to
> just changing the code without a catversion bump.  OTOH, if we want to
> encourage testing of pg_upgrade ...

FWIW, the last bump has led to a lot of testing of pgupgrade.

-- 
                                  -- Josh Berkus
                                     PostgreSQL Experts Inc.
                                     http://www.pgexperts.com

  11. Re: testing plpython3u on 9.0beta2

    Andres Freund <andres@anarazel.de> — 2010-06-25T21:44:46Z

    On Wednesday 23 June 2010 16:30:54 Tom Lane wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
> > I can reproduce this, here.  The problem seems to be that plpython
> > only build either plpython2.so or plython3.so, but both languages
> > expect a call handler called plython_call_handler.  So once we load
> > the shared library for one language, the other language just grabs the
> > same call handler.
> 
> We had better fix that --- I can definitely foresee installations
> wanting to use both plpython2 and plpython3.  This'd require a change in
> the default contents of pg_pltemplate, though.  Does it seem important
> enough to bump catversion for?
Has anybody actually researched if it is safe to run python2 and python3 in 
the same address space? I wouldnt be surprised at all if that where 
problematic.

Andres

  12. Re: testing plpython3u on 9.0beta2

    Peter Eisentraut <peter_e@gmx.net> — 2010-06-25T22:57:54Z

    On fre, 2010-06-25 at 23:44 +0200, Andres Freund wrote:
> Has anybody actually researched if it is safe to run python2 and
> python3 in the same address space?

You can't run plpython2 and plpython3 in the same session, because the
libraries are loaded with dlopen(RTLD_GLOBAL) (with RTLD_LOCAL it would
apparently work).  But you can use them in different sessions on the
same database, for example.

  13. Re: testing plpython3u on 9.0beta2

    Robert Haas <robertmhaas@gmail.com> — 2010-06-28T12:14:10Z

    On Fri, Jun 25, 2010 at 2:49 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
> On fre, 2010-06-25 at 10:17 -0400, Tom Lane wrote:
>> Peter Eisentraut <peter_e@gmx.net> writes:
>> > The problem is apparently that when CREATE LANGUAGE creates a language
>> > from a pg_pltemplate entry, it creates the proname from the tmplhandler
>> > name, and if it finds a fitting proname entry already, it used that one.
>> > So when you create plpython2 first and plpython3 second, the pg_language
>> > entries of the latter point to the pg_proc entries of the former.
>>
>> > If you fix that up manually (create additional pg_proc entries and fix
>> > the pg_language entries to point there), it works better.
>>
>> The fix ought to be to change the function nmes used by plpython3 ...
>
> Right.  What shall we do about the catversion?

We should go ahead and apply this, either with (my vote) or without
(Tom's vote) a catversion bump.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

  14. Re: testing plpython3u on 9.0beta2

    Bruce Momjian <bruce@momjian.us> — 2010-06-28T21:18:29Z

    Josh Berkus wrote:
> 
> > You could argue it either way.  The number of beta testers with
> > plpython3 installations is probably very small, so I'm kinda leaning to
> > just changing the code without a catversion bump.  OTOH, if we want to
> > encourage testing of pg_upgrade ...
> 
> FWIW, the last bump has led to a lot of testing of pgupgrade.

And fixes, that will appear in 9.0 beta3.  :-)  Most fixes were related
to platform compile portability.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + None of us is going to be here forever. +

  15. Re: testing plpython3u on 9.0beta2

    Tom Lane <tgl@sss.pgh.pa.us> — 2010-06-28T22:54:16Z

    Bruce Momjian <bruce@momjian.us> writes:
> Josh Berkus wrote:
>>> You could argue it either way.  The number of beta testers with
>>> plpython3 installations is probably very small, so I'm kinda leaning to
>>> just changing the code without a catversion bump.  OTOH, if we want to
>>> encourage testing of pg_upgrade ...
>> 
>> FWIW, the last bump has led to a lot of testing of pgupgrade.

> And fixes, that will appear in 9.0 beta3.  :-)  Most fixes were related
> to platform compile portability.

Well, if you think that pg_upgrade has changed materially since beta2,
that would be a good argument for getting some fresh testing for it,
which in turn argues for doing the catversion bump here.

			regards, tom lane

  16. Re: testing plpython3u on 9.0beta2

    Bruce Momjian <bruce@momjian.us> — 2010-06-28T22:59:41Z

    Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > Josh Berkus wrote:
> >>> You could argue it either way.  The number of beta testers with
> >>> plpython3 installations is probably very small, so I'm kinda leaning to
> >>> just changing the code without a catversion bump.  OTOH, if we want to
> >>> encourage testing of pg_upgrade ...
> >> 
> >> FWIW, the last bump has led to a lot of testing of pgupgrade.
> 
> > And fixes, that will appear in 9.0 beta3.  :-)  Most fixes were related
> > to platform compile portability.
> 
> Well, if you think that pg_upgrade has changed materially since beta2,
> that would be a good argument for getting some fresh testing for it,
> which in turn argues for doing the catversion bump here.

Attached are the changes since beta2;  they are pretty minor.  The good
news is I think all reporters eventually got it working.  I assume using
it for beta3 would allow it work even better, and once you have use it
once, using it again is simple.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + None of us is going to be here forever. +

  17. Re: testing plpython3u on 9.0beta2

    Peter Eisentraut <peter_e@gmx.net> — 2010-06-29T00:19:28Z

    On fre, 2010-06-25 at 18:57 -0400, Peter Eisentraut wrote:
> On fre, 2010-06-25 at 23:44 +0200, Andres Freund wrote:
> > Has anybody actually researched if it is safe to run python2 and
> > python3 in the same address space?
> 
> You can't run plpython2 and plpython3 in the same session, because the
> libraries are loaded with dlopen(RTLD_GLOBAL) (with RTLD_LOCAL it would
> apparently work).  But you can use them in different sessions on the
> same database, for example.

By the way, I thought there had been some discussion about this in the
past, but I couldn't find it ...

Why are we using RTLD_GLOBAL?

  18. Re: testing plpython3u on 9.0beta2

    Tom Lane <tgl@sss.pgh.pa.us> — 2010-06-29T00:45:35Z

    Peter Eisentraut <peter_e@gmx.net> writes:
> Why are we using RTLD_GLOBAL?

Some guy named Eisentraut wanted it for plpython:
http://archives.postgresql.org/pgsql-hackers/2001-05/msg00563.php
http://archives.postgresql.org/pgsql-committers/2001-05/msg00283.php

Possibly that no longer applies, though.  In general it seems like
our usage of loadable modules treats them all as independent objects,
so that not using RTLD_GLOBAL would be a more sensible policy.  If it
won't break any of the existing PLs then I'm fine with removing it.

			regards, tom lane