Thread
-
plperl intial pass
Mark Hollomon <mhh@mindspring.com> — 1999-07-23T20:15:10Z
on the patches list, I posted the first files for suuport for using perl as the procedural language. The makefile uses libtool (probably incorrectly). It is not 'safe'. Perl XS modules cannot be used. Using Perl in rules hasn't even been thought about. IF the code looks suspiciously like Jan's code for pltcl - well, it _is_ Jan's code for pltcl horribly mangled. It compiles and runs for me. YMMV. more code and docs to follow. -- Mark Hollomon mhh@mindspring.com
-
Re: [HACKERS] plperl intial pass
Oleg Bartunov <oleg@sai.msu.su> — 1999-07-24T18:26:33Z
Great, just compiled and install but need to look at some examples :-) btw, here is a patch for createlang command to enable plperl I'm not sure about trusted field. --- createlang Sat Jul 24 22:27:05 1999 +++ /usr/local/pgsql/bin/createlang Wed Jul 21 19:36:55 1999 @@ -84,9 +84,6 @@ plpgsql) lancomp="PL/pgSQL" trusted="TRUSTED" handler="plpgsql_call_handler";; - plperl) lancomp="PL/Perl" - trusted="TRUSTED" - handler="plperl_call_handler";; pltcl) lancomp="PL/Tcl" trusted="TRUSTED" handler="pltcl_call_handler";; On Fri, 23 Jul 1999, Mark Hollomon wrote: > Date: Fri, 23 Jul 1999 16:15:10 -0400 > From: Mark Hollomon <mhh@mindspring.com> > To: pgsql-hackers@postgreSQL.org > Subject: [HACKERS] plperl intial pass > > on the patches list, I posted the first files > for suuport for using perl as the procedural > language. > > The makefile uses libtool (probably incorrectly). > > It is not 'safe'. > > Perl XS modules cannot be used. > > Using Perl in rules hasn't even been thought about. > > IF the code looks suspiciously like Jan's code > for pltcl - well, it _is_ Jan's code for pltcl > horribly mangled. > > It compiles and runs for me. YMMV. > > more code and docs to follow. > -- > Mark Hollomon > mhh@mindspring.com > _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 -
Re: [HACKERS] plperl intial pass
Mark Hollomon <mhh@mindspring.com> — 1999-07-25T16:08:51Z
On Sat, Jul 24, 1999 at 10:26:33PM +0400, Oleg Bartunov wrote: > Great, > > just compiled and install but need to look at some examples :-) > btw, here is a patch for createlang command to enable plperl > I'm not sure about trusted field. At this point it definitely should not be trusted. (Trust me). How about the famous hello world: create function hello () returns text as ' return "Hello world!";' language 'plperl'; or a quick sum: create function sum2 (int4, int4) returns int4 as ' $_[0] + $_[1];' language 'plperl'; The args are in @_ (naturally). Tuples are passed as hash references. Access to SPI functionality is coming. -- Mark Hollomon mhh@mindspring.com
-
Re: [HACKERS] plperl intial pass
Oleg Bartunov <oleg@sai.msu.su> — 1999-07-26T00:28:39Z
Great ! at least your examples works: test=> select hello(); NOTICE: plperl_func_handler: have been asked to call __PLperl_proc_329289 NOTICE: plperl_func_handler: __PLperl_proc_329289 was in the hash NOTICE: plperl_call_perl_func: calling __PLperl_proc_329289 NOTICE: plperl_func_handler: returned from plperl_call_perl_func NOTICE: plperl_func_handler: return as string = Hello world! NOTICE: plperl_func_handler: Datum is 826ee30 hello ------------ Hello world! (1 row) test=> create function sum2 (int4, int4) returns int4 as ' test'> $_[0] + $_[1];' language 'plperl'; CREATE test=> select sum2(4,2); NOTICE: plperl_func_handler: have been asked to call __PLperl_proc_329290 NOTICE: plperl_func_handler: __PLperl_proc_329290 doesn't exist yet NOTICE: plperl_create_sub: creating the sub NOTICE: plperl_call_perl_func: calling __PLperl_proc_329290 NOTICE: plperl_func_handler: returned from plperl_call_perl_func NOTICE: plperl_func_handler: return as string = 6 NOTICE: plperl_func_handler: Datum is 6 sum2 ---- 6 (1 row) Regards, Oleg On Sun, 25 Jul 1999, Mark Hollomon wrote: > Date: Sun, 25 Jul 1999 12:08:51 -0400 > From: Mark Hollomon <mhh@mindspring.com> > To: Oleg Bartunov <oleg@sai.msu.su> > Cc: pgsql-hackers@postgreSQL.org > Subject: Re: [HACKERS] plperl intial pass > > On Sat, Jul 24, 1999 at 10:26:33PM +0400, Oleg Bartunov wrote: > > Great, > > > > just compiled and install but need to look at some examples :-) > > btw, here is a patch for createlang command to enable plperl > > I'm not sure about trusted field. > > At this point it definitely should not be trusted. (Trust me). > > How about the famous hello world: > > create function hello () returns text as ' > return "Hello world!";' language 'plperl'; > > or a quick sum: > > create function sum2 (int4, int4) returns int4 as ' > $_[0] + $_[1];' language 'plperl'; > > The args are in @_ (naturally). Tuples are passed as > hash references. > > Access to SPI functionality is coming. > > > > -- > Mark Hollomon > mhh@mindspring.com > _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
-
Re: [HACKERS] plperl intial pass
Dmitry Samersoff <dms@wplus.net> — 1999-07-26T08:19:56Z
On 24-Jul-99 Oleg Bartunov wrote: > Great, > > just compiled and install but need to look at some examples :-) > btw, here is a patch for createlang command to enable plperl > I'm not sure about trusted field. Does plperl use Perl interpreter or it's completly different language with similar syntax? Do you have some speed/memory statistic or plpgsql/plperl comparison ? --- Dmitry Samersoff, dms@wplus.net, ICQ:3161705 http://devnull.wplus.net * There will come soft rains ...
-
Re: [HACKERS] plperl intial pass
Jan Wieck <wieck@debis.com> — 1999-07-26T09:15:25Z
> > Great, > > just compiled and install but need to look at some examples :-) > btw, here is a patch for createlang command to enable plperl > I'm not sure about trusted field. > > --- createlang Sat Jul 24 22:27:05 1999 > +++ /usr/local/pgsql/bin/createlang Wed Jul 21 19:36:55 1999 > @@ -84,9 +84,6 @@ > plpgsql) lancomp="PL/pgSQL" > trusted="TRUSTED" > handler="plpgsql_call_handler";; > - plperl) lancomp="PL/Perl" > - trusted="TRUSTED" > - handler="plperl_call_handler";; > pltcl) lancomp="PL/Tcl" > trusted="TRUSTED" > handler="pltcl_call_handler";; I wouldn't make it a TRUSTED language right now, because until PL/Perl has a safe mode (what Mark said it hasn't now) it is a security hole. Unpriviliged users could create functions in PL/Perl that modify the hba.conf! Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #========================================= wieck@debis.com (Jan Wieck) # -
Re: [HACKERS] plperl intial pass
Jan Wieck <wieck@debis.com> — 1999-07-26T09:50:52Z
Dmitry Samersoff wrote: > > On 24-Jul-99 Oleg Bartunov wrote: > > Great, > > > > just compiled and install but need to look at some examples :-) > > btw, here is a patch for createlang command to enable plperl > > I'm not sure about trusted field. > > Does plperl use Perl interpreter or it's completly different language > with similar syntax? > > Do you have some speed/memory statistic or plpgsql/plperl comparison ? It uses a real Perl precompiler/interpreter inside. I think it's far too early for such comparisions. As Mark wrote, PL/Perl's SPI interface (for accessing tables from inside a function) is still to come, and if I remember right, triggers are another delayed feature up to now. When it's done, I would expect that PL/Perl could outperform PL/pgSQL in many cases. I haven't done speed comparision between PL/pgSQL and PL/Tcl yet, but I know all their internals. The reason for my assumtion is that PL/pgSQL uses the PostgreSQL executor for all computations. That's IMHO a pro, because it assures that any defined datatype, function, operator and aggregate is automagically available in PL/pgSQL and all computations return exactly the same result as if they're done inside an SQL statement. But nothing on earth is for free, not even the death - you pay for it with your life. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #========================================= wieck@debis.com (Jan Wieck) # -
Re: [HACKERS] plperl intial pass
Jan Wieck <wieck@debis.com> — 1999-07-26T10:18:06Z
Mark Hollomon wrote: > > on the patches list, I posted the first files > for suuport for using perl as the procedural > language. Congratulations! > > The makefile uses libtool (probably incorrectly). > > It is not 'safe'. > > Perl XS modules cannot be used. What's an XS module? If it's a shared object dynamically linked - don't care too much - PL/Tcl cannot either. > > Using Perl in rules hasn't even been thought about. If a function works from a query, the same function must work too in a rule because the rewriter only mangles up parsetrees so when executing, they are invoked from a query. Or did you mean triggers? > > IF the code looks suspiciously like Jan's code > for pltcl - well, it _is_ Jan's code for pltcl > horribly mangled. What ya think where the skeleton for PL/pgSQL came from :-) I just wrote my own SQL scripting bytecode compiler and executor and placed them into the PL/Tcl sources. Congrats again - great work - move on. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #========================================= wieck@debis.com (Jan Wieck) # -
Re: [HACKERS] plperl intial pass
Mark Hollomon <mhh@nortelnetworks.com> — 1999-07-26T13:48:38Z
Dmitry Samersoff wrote: > > On 24-Jul-99 Oleg Bartunov wrote: > > Great, > > > > just compiled and install but need to look at some examples :-) > > btw, here is a patch for createlang command to enable plperl > > I'm not sure about trusted field. > > Does plperl use Perl interpreter or it's completly different language > with similar syntax? > It imbeds the perl interpreter. Just as pltcl imbeds the tcl interpreter. > Do you have some speed/memory statistic or plpgsql/plperl comparison ? No. -- Mark Hollomon mhh@nortelnetworks.com ESN 451-9008 (302)454-9008
-
Re: [HACKERS] plperl intial pass
Mark Hollomon <mhh@nortelnetworks.com> — 1999-07-26T13:59:54Z
Jan Wieck wrote: > > Mark Hollomon wrote: > > > > > It is not 'safe'. > > > > Perl XS modules cannot be used. > > What's an XS module? If it's a shared object dynamically > linked - don't care too much - PL/Tcl cannot either. Correct. The problem is that the Opcode module, which allows you to disable features of the compiler (to close security holes) is an XS module. In theory, it is possible to do without Opcode, but doing so would create a very heavy perl version dependency in plperl. So, I have to get XS stuff working in order to disallow XS stuff. sigh. And plperl can never be trusted until I can forbid writing to the filesystem. > > > > > Using Perl in rules hasn't even been thought about. > > If a function works from a query, the same function must work > too in a rule because the rewriter only mangles up parsetrees > so when executing, they are invoked from a query. Or did you > mean triggers? Ick. Correct. I meant triggers. -- Mark Hollomon mhh@nortelnetworks.com ESN 451-9008 (302)454-9008
-
Re: [HACKERS] plperl intial pass
Jan Wieck <wieck@debis.com> — 1999-07-26T15:45:19Z
Mark Hollomon wrote: > > Jan Wieck wrote: > > > > Correct. The problem is that the Opcode module, which allows you to > disable features of the compiler (to close security holes) is an > XS module. In theory, it is possible to do without Opcode, but > doing so would create a very heavy perl version dependency in plperl. > > So, I have to get XS stuff working in order to disallow XS stuff. > sigh. > > And plperl can never be trusted until I can forbid writing to the > filesystem. I see. Maybe it's possible to get the Opcode stuff working without full XS? Adding full XS support only to disable it - what an overkill :-) Correct me if I'm wrong (I'm only guessing). Like for Perl, the Tcl interpreter itself sits in a library. To create the standalone tclsh, a small tclAppInit.c file is compiled into the tclsh executable. The default one only creates one interpreter and arranges for the execution of the script given in argv[0] or starts up the interactive shell. A dynamically loadable Tcl module contains one special function named <libname>_Init() where first character of libname is capitalized. On dynamic load, this function is called with the invoking interpreter as argument. This function then calls Tcl_CreateCommand() etc. to tell Tcl what's coming here and does other module specific initializations. It is now possible, to add other stuff to tclAppInit.c (like calls to Mymodule_Init) and link it against some more than libtcl.so. That was the standard solution before dynamic loading was that easy as it is today (back in the days of a.out libs). Your plperl.c is mostly my pltcl.c - so I assume it does the same things mainly. Create an interpreter and throw some strings into it, hoping they are intelligable in some way (at least produce a helpful error message). Thus, it might be possible to add calls to the initializations for the Opcode XS directly into the plperl module after creating the interpreter and link it against Opcode as well. This is just the way I would do it for Tcl and I'll surely do it someday. I would like to have a second, unsafe interpreter in the module. That could then modify files or use the frontend library to access a different database on another server. Needless to say that this then would be an untrusted language, available only for db superusers. Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #========================================= wieck@debis.com (Jan Wieck) # -
Re: [HACKERS] plperl intial pass
Mark Hollomon <mhh@nortelnetworks.com> — 1999-07-26T17:38:36Z
Jan Wieck wrote: > > Mark Hollomon wrote: > > > > Jan Wieck wrote: > > > > > > > Correct. The problem is that the Opcode module, which allows you to > > disable features of the compiler (to close security holes) is an > > XS module. In theory, it is possible to do without Opcode, but > > doing so would create a very heavy perl version dependency in plperl. > > > > So, I have to get XS stuff working in order to disallow XS stuff. > > sigh. > > > > And plperl can never be trusted until I can forbid writing to the > > filesystem. > > I see. Maybe it's possible to get the Opcode stuff working > without full XS? Adding full XS support only to disable it - > what an overkill :-) > > Correct me if I'm wrong (I'm only guessing). Like for Perl, > the Tcl interpreter itself sits in a library. To create the > standalone tclsh, a small tclAppInit.c file is compiled into > the tclsh executable. The default one only creates one > interpreter and arranges for the execution of the script > given in argv[0] or starts up the interactive shell. > > A dynamically loadable Tcl module contains one special > function named <libname>_Init() where first character of > libname is capitalized. On dynamic load, this function is > called with the invoking interpreter as argument. This > function then calls Tcl_CreateCommand() etc. to tell Tcl ^^^^^^^^^^^^^^^^^ And here-in lies the problem. Tcl_CreateCommand is sitting, not in the executable, but in the shared-lib with the function call handler. dlopen(), by default will not link across shared-libs. postgres /-----/ \-----\ | | plperl.so ---> Opcode.so ^^ This link doesn't happen. Passing RTLD_GLOBAL (I think) as a flag to dlopen makes the symbols in a shared-lib available for linking into the next shared-lib. But postgresql doesn't use the RTLD_GLOBAL flag and patching the backend to load _everything_ with RTLD_GLOBAL seemed like it could have less than desirable behavior. a.out systems are easier since perl's dynamic loading subsystem would take care of problem for me. > what's coming here and does other module specific > initializations. > > It is now possible, to add other stuff to tclAppInit.c (like > calls to Mymodule_Init) and link it against some more than > libtcl.so. That was the standard solution before dynamic > loading was that easy as it is today (back in the days of > a.out libs). That is exactly how it works. But see above. And on top of the above problem, postgres assumes all linuxen use a.out type loading. Where as perl uses dlopen where it can. Getting those two to play together is more than I care to attempt. I am researching a fix now to let linux installations use dlopen if it is available. I would not be unhappy if somebody beats me to it. > This is just the way I would do it for Tcl and I'll surely do > it someday. I would like to have a second, unsafe > interpreter in the module. That could then modify files or > use the frontend library to access a different database on > another server. Needless to say that this then would be an > untrusted language, available only for db superusers. > Yes, I've been thinking about that as well. It would be nice to have permissions based on userid. Maybe the 'suid' stuff that is being discussed in another thread will gives us a mechanism. -- Mark Hollomon mhh@nortelnetworks.com ESN 451-9008 (302)454-9008 -
Re: [HACKERS] plperl intial pass
Jan Wieck <wieck@debis.com> — 1999-07-26T23:01:44Z
Mark Hollomon wrote: > > A dynamically loadable Tcl module contains one special > > function named <libname>_Init() where first character of > > libname is capitalized. On dynamic load, this function is > > called with the invoking interpreter as argument. This > > function then calls Tcl_CreateCommand() etc. to tell Tcl > ^^^^^^^^^^^^^^^^^ > > And here-in lies the problem. Tcl_CreateCommand is sitting, not > in the executable, but in the shared-lib with the function call > handler. dlopen(), by default will not link across shared-libs. > > postgres > /-----/ \-----\ > | | > plperl.so ---> Opcode.so > ^^ > This link doesn't happen. But it does for PL/Tcl - at least under Linux-ELF. (C = Call to, L = Location of functions code segment): +-------------------------+ | postgres | +-------------------------+ | | dynamic load | v +---------------------------+ +---------------------------+ | pltcl.so |--------->| libtcl8.0.so | | | auto- | | | C Tcl_CreateInterp() | dynamic | L Tcl_CreateInterp() | | C Tcl_CreateCommand() | load | L Tcl_CreateCommand() | | L static pltcl_SPI_exec() | | C pltcl_SPI_exec() | +---------------------------+ +---------------------------+ After loading of pltcl.so, it calls Tcl_CreateInterp() to build a Tcl interpreter, and then calls Tcl_CreateCommand() to tell that interpreter the address of one of it's hidden (static) functions plus a name for it from the script side. The interpreter just remembers this in it's command hash table, and if that keyword occurs when it expects a command/procedure name, just calls it via the function pointer. There is no -ltcl8.0 switch in the link step of postgres. The fact that pltcl.so needs something out of libtcl8.0.so is told when linking pltcl.so: gcc -shared -o pltcl.so pltcl.o -L/usr/local/lib -ltcl8.0 That results in this: [pgsql@hot] ~ > ldd bin/postgres libdl.so.1 => /lib/libdl.so.1 (0x4000a000) libm.so.5 => /lib/libm.so.5 (0x4000d000) libtermcap.so.2 => /usr/lib/libtermcap.so.2 (0x40016000) libncurses.so.3.0 => /lib/libncurses.so.3.0 (0x4001a000) libc.so.5 => /lib/libc.so.5 (0x4005b000) [pgsql@hot] ~ > ldd lib/pltcl.so ./lib/pltcl.so => ./lib/pltcl.so (0x4000a000) libc.so.5 => /lib/libc.so.5 (0x40010000) libtcl8.0.so => /usr/local/lib/libtcl8.0.so (0x400cb000) As you see, there is no libtcl mentioned in the shared lib dependencies of the postgres backend. It's the pltcl.so shared object that remembers this. And if you invoke "ldd -r -d pltcl.so" it will print alot of unresolveable symbols, but most of them are backend symbols (the others are math ones because the above gcc -shared call is in fact incomplete - but since the backend is already linked against libm.so it doesn't matter :-). So if I want to use My dynamically loadable package for Tcl from inside the PL/Tcl interpreter, I would have to call My_Init() from pltcl.so AND add My.so to the linkage of pltcl.so. Calling My_Init() causes that "pltcl.o" has an unresolved reference to symbol _My_Init. The linker find's it in My.so and saves this info in pltcl.so so the dynamic loader can (and does) resolve it whenever something load pltcl.so. The important key is to reference at least one symbol in the shared lib you want to get automatically loaded. You can add as much link libs with -l as you want. If none of their symbols is needed, the linker will not save this dependency (because there is none) in the resulting .so. I'll give it a try and USE some binary Tcl packages from inside. Will tell ya soon. > Getting those two to play together is more than I care to attempt. > I am researching a fix now to let linux installations use dlopen > if it is available. Don't think you need to. > > This is just the way I would do it for Tcl and I'll surely do > > it someday. I would like to have a second, unsafe > > interpreter in the module. That could then modify files or > > use the frontend library to access a different database on > > another server. Needless to say that this then would be an > > untrusted language, available only for db superusers. > > > > Yes, I've been thinking about that as well. It would be nice to have > permissions based on userid. Maybe the 'suid' stuff that is being > discussed in another thread will gives us a mechanism. I know, I know - and I know how. It cannot work for "internal" language functions. But for anything that goes through some loading (dynloader or PL call hander), the fmgr looks up pg_proc and put's informations into the FmgrInfo struct. Adding a setuid field to pg_proc and remembering that too wouldn't be too much and it then would know when calling such a beast. Fmgr then manages a current user stack which must be reset on a transaction abort. Anything that needs the current user simply looks at the toplevel stack entry. This is totally transparent then for all non-builtin functions and all non-builtin triggers (where I don't know of one). Maybe I kept this far too long in mind. But I thought about some more complicated changes to the function call interface for a while that would require touching several dozens of source files (single argument NULL identification, returning tuples and tuple SET's). Doing SETUID would have been some DONE WHILE AT IT. I really should do it earlier than the SET's, because they require subselecting RTE's (which it the third thread now - eh - I better shut up). Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #========================================= wieck@debis.com (Jan Wieck) # -
dynloader and PLs [was: plperl intial pass]
Mark Hollomon <mhh@nortelnetworks.com> — 1999-07-27T12:54:47Z
Jan Wieck wrote: > > Mark Hollomon wrote: > > > Jan Wieck wrote: > > > > > > A dynamically loadable Tcl module contains one special > > > function named <libname>_Init() where first character of > > > libname is capitalized. On dynamic load, this function is > > > called with the invoking interpreter as argument. This > > > function then calls Tcl_CreateCommand() etc. to tell Tcl > > ^^^^^^^^^^^^^^^^^ > > > > And here-in lies the problem. Tcl_CreateCommand is sitting, not > > in the executable, but in the shared-lib with the function call > > handler. dlopen(), by default will not link across shared-libs. > > > > postgres > > /-----/ \-----\ > > | | > > plperl.so ---> Opcode.so > > ^^ > > This link doesn't happen. > > But it does for PL/Tcl - at least under Linux-ELF. (C = Call > to, L = Location of functions code segment): > > +-------------------------+ > | postgres | > +-------------------------+ > | > | dynamic load > | > v > +---------------------------+ +---------------------------+ > | pltcl.so |--------->| libtcl8.0.so | > | | auto- | | > | C Tcl_CreateInterp() | dynamic | L Tcl_CreateInterp() | > | C Tcl_CreateCommand() | load | L Tcl_CreateCommand() | > | L static pltcl_SPI_exec() | | C pltcl_SPI_exec() | > +---------------------------+ +---------------------------+ > > After loading of pltcl.so, it calls Tcl_CreateInterp() to > build a Tcl interpreter, and then calls Tcl_CreateCommand() > to tell that interpreter the address of one of it's hidden > (static) functions plus a name for it from the script side. > The interpreter just remembers this in it's command hash > table, and if that keyword occurs when it expects a > command/procedure name, just calls it via the function > pointer. AHHH, now I understand the difference. By default, the perl installation does not create a shared library. It creates a static archive only. And the three linux distros that I have experience with don't force the creation of the shared lib. So, my situation is: postgres | | +----------------------+ +-----------------+ | plperl.so | | Opcode.so | | +--------------+ | | | | | libperl.a | <-+------------| | | +--------------+ | | | +----------------------+ +-----------------+ And it is THAT link that I cannot get to happen without the RTLD_GLOBAL flag I mentioned. Sorry for the confusion. Hopefully you can help find a way out of this. I had a patch to change the way dynloader worked on linuxelf, but over night my disk crashed. brand new UDMA/66 drive. Grrrr. -- Mark Hollomon mhh@nortelnetworks.com ESN 451-9008 (302)454-9008 -
fmgr interface [was: plperl inital pass]
Mark Hollomon <mhh@nortelnetworks.com> — 1999-07-27T13:01:01Z
Jan Wieck wrote: > > Mark Hollomon wrote: > > > Yes, I've been thinking about that as well. It would be nice to have > > permissions based on userid. Maybe the 'suid' stuff that is being > > discussed in another thread will gives us a mechanism. > > I know, I know - and I know how. It cannot work for > "internal" language functions. But for anything that goes > through some loading (dynloader or PL call hander), the fmgr > looks up pg_proc and put's informations into the FmgrInfo > struct. Adding a setuid field to pg_proc and remembering that > too wouldn't be too much and it then would know when calling > such a beast. Fmgr then manages a current user stack which > must be reset on a transaction abort. Anything that needs the > current user simply looks at the toplevel stack entry. That would work. > > This is totally transparent then for all non-builtin > functions and all non-builtin triggers (where I don't know of > one). > > Maybe I kept this far too long in mind. But I thought about > some more complicated changes to the function call interface > for a while that would require touching several dozens of > source files (single argument NULL identification, returning > tuples and tuple SET's). Doing SETUID would have been some > DONE WHILE AT IT. I really should do it earlier than the > SET's, because they require subselecting RTE's (which it the > third thread now - eh - I better shut up). I've been looking at returning a tuple. It looked to me that the executor would handle a returned tuple okay, it was just SETs that would cause problems. But I suspect I am wrong. The best I could come up with for creating the tuple was using heap_formtuple. But that requires a TupleDesc so I was going to use heap_openr. But that needs the name of the relation which is avaible from the Form_pg_data (?) structure for the return type, which we already must get. -- Mark Hollomon mhh@nortelnetworks.com ESN 451-9008 (302)454-9008
-
Re: fmgr interface [was: plperl inital pass]
Jan Wieck <wieck@debis.com> — 1999-07-27T14:39:27Z
Mark Hollomon wrote: > I've been looking at returning a tuple. It looked to me that the > > executor would handle a returned tuple okay, it was just SETs that > would cause problems. But I suspect I am wrong. Functions returning SET's allways return SET's of tuples, never SET's of single values. And functions returning tuple (SET's) have a targetlist to specify which attribute of the returned tuple(s) is wanted. It is the processing of this funciton-call-targetlist that's actually broken in the executor. But it's not worth fixing it without beeing able after to use more than one attribute of the returned set. And that requires the mentioned subselecting RTE. So you could then say things like: SELECT X.a, X.c FROM mysetfunc('Mark') X; The next problem in returning SET's is, that PostgreSQL isn't a state machine - it is stack oriented. The way it was supposed to work with SQL language functions was this: 1. The last query in an SQL function returning a tuple SET is allways a SELECT. 2. When the FUNC node is first hit during execution, the function is called. Then the FUNC node is modified by the executor and references the execution tree of the last command in the function. 3. Subsequent function calls don't invoke the function again, instead functions last commands execution tree is asked for the next tuple. This mechanism could also work for PL functions. A PL function returning a SET creates a temp table. At each occurence of RETURN mytup AND RESUME; it adds the tuple to the temp table. If it finally really returns, it hands back an execution plan for a SELECT * FROM <my_invocations_temp_table>; Then again, the problem of using multiple attributes of the returned set remains. > The best I could come up with for creating the tuple was using > > heap_formtuple. But that requires a TupleDesc so I was going to > > use heap_openr. But that needs the name of the relation which is > > avaible from the Form_pg_data (?) structure for the return type, > > which we already must get. Of course, the PL function must create tuples via heap_formtuple(). Thus, we need a pg_class entry (etc.) for it. The PL handler knows the return type of the function it's handling from pg_proc. The corresponding pg_type entry has a non-zero typrelid indicating that it's a tuple type. Simply use heap_open() with that typrelid and you'll get it. I'd like to add a new type of relation when we go for return SET's. CREATE STRUCTURE structname (attname type [, ...]); It just causes another pg_class entry, but these relations aren't accessible by normal means and do not have an underlying file. Don't know if it's valid SQL syntax, but what else could tell the parser what type of a tuple a SET function will return if it's not an existing relation structure? Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #========================================= wieck@debis.com (Jan Wieck) # -
Re: dynloader and PLs [was: plperl intial pass]
Jan Wieck <wieck@debis.com> — 1999-07-27T17:05:28Z
Mark Hollomon wrote: > AHHH, now I understand the difference. By default, the perl installation > does not create a shared library. It creates a static archive only. > And the three linux distros that I have experience with don't force > the creation of the shared lib. So, my situation is: > > postgres > | > | > +----------------------+ +-----------------+ > | plperl.so | | Opcode.so | > | +--------------+ | | | > | | libperl.a | <-+------------| | > | +--------------+ | | | > +----------------------+ +-----------------+ > > And it is THAT link that I cannot get to happen without the RTLD_GLOBAL > flag I mentioned. Yes - we need to understand the differences. After looking at some perl manpages (perlxs, perlembed, perlmodlib etc.) and consulting Opcode.pm I see the problems clearer now. Under Tcl, you can simply type "load <shared-object>" to load a .so and cause a call to it's ..._Init() function. Whatever comes there, the .so's ..._Init() function will tell it. And since every C function that should be callable from Tcl is given to the interpreter as a function pointer from within the ..._Init(), nothing except the ..._Init() function itself must be really resolved. In fact, the functions called from Tcl can be declared static inside the shared object (what's true in pltcl) so there are no symbols to resolve. A safe Tcl interpreter has no load command. But the controlling C application can call the .so's ..._Init() function directly to simulate the "load" (well, it should be the ..._SafeInit(), but that's another story). Thus, a C application creating a safe interpreter can load modules for it even if the interpreter itself can't. Under Perl, a package using a shared object is allways surrounded by some .pm which tells to lookup symbols via the dynamic loader (if I understand XSUB's right). So it's still a type of a script that controls the Perl->C-function bindings, not the shared object itself. The detail I don't understand is what breaks Perl's dynaloader if you use it from inside of plperl.so. Since Perl isn't built shared, the entire libperl.a should be linked static into plperl.so. What's the exact error message when you try to USE Opcode? > > Sorry for the confusion. > > Hopefully you can help find a way out of this. > > I had a patch to change the way dynloader worked on linuxelf, I don't think you should change the entire dynamic loader of PostgreSQL for it. This could be a can of worms and you should be happy that these problems showed up already on your development platform. I don't expect that you're willing to fix the dynamic loading under AIX, HP-UX and Solaris too (maybe you can't because the lack of appropriate environment). > but over night my disk crashed. brand new UDMA/66 drive. Grrrr. Ech Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #========================================= wieck@debis.com (Jan Wieck) # -
Re: dynloader and PLs [was: plperl intial pass]
Mark Hollomon <mhh@nortelnetworks.com> — 1999-07-27T18:06:25Z
Jan Wieck wrote: > > Mark Hollomon wrote: > > > I had a patch to change the way dynloader worked on linuxelf, > > I don't think you should change the entire dynamic loader of > PostgreSQL for it. This could be a can of worms and you > should be happy that these problems showed up already on your > development platform. I don't expect that you're willing to > fix the dynamic loading under AIX, HP-UX and Solaris too > (maybe you can't because the lack of appropriate > environment). > The problem is that perl and postgres disagree as how to do the dynamic loading. postgres (on linux) _Always_ use aout style dynamic loading. Perl checks to see if the system is ELF and use dlopen if it is. On my ELF system then, postgres is loading plperl.so with dl_open (?). Then perl is loading Opcode.so using dlopen. The problem seems to be that the symbols from libperl.a (in plperl.so) are not available for resolving missing symbols in Opcode.so. The error message basically mentions every perl symbol as 'unresolved'. I noticed in another thread that D'Arcy is strugling with a similar problem in NetBSD. On my system, once I got postgres and perl to agree on how to do dynamic loading, I got XS stuff working. The code is (mostly) already in plperl.c, but ifdef'ed out. -- Mark Hollomon mhh@nortelnetworks.com ESN 451-9008 (302)454-9008