Thread
-
PL/Python array support
Peter Eisentraut <peter_e@gmx.net> — 2009-11-04T14:02:08Z
Here is a patch to support arrays in PL/Python as parameters and return values. It converts an array parameter to a Python "list", and converts a Python "sequence" return value back to an array. I have settled on two implementation restrictions for the moment: - Only supports one-dimensional arrays. (Python has no multidimensional lists, so the semantics of this would be dubious.) - Does not support returning arrays of composite types. (Basically too complicated to implement right now and seemingly of limited practical value.)
-
Re: PL/Python array support
Robert Haas <robertmhaas@gmail.com> — 2009-11-04T14:44:10Z
On Wed, Nov 4, 2009 at 9:02 AM, Peter Eisentraut <peter_e@gmx.net> wrote: > Here is a patch to support arrays in PL/Python as parameters and return > values. It converts an array parameter to a Python "list", and converts > a Python "sequence" return value back to an array. This is probably a stupid question, but why would you use different types for incoming and outgoing data flow? ...Robert
-
Re: PL/Python array support
Peter Eisentraut <peter_e@gmx.net> — 2009-11-04T16:06:38Z
On ons, 2009-11-04 at 09:44 -0500, Robert Haas wrote: > On Wed, Nov 4, 2009 at 9:02 AM, Peter Eisentraut <peter_e@gmx.net> wrote: > > Here is a patch to support arrays in PL/Python as parameters and return > > values. It converts an array parameter to a Python "list", and converts > > a Python "sequence" return value back to an array. > > This is probably a stupid question, but why would you use different > types for incoming and outgoing data flow? A list is one particular kind of sequence. See also http://docs.python.org/reference/datamodel.html#the-standard-type-hierarchy
-
Re: PL/Python array support
Peter Eisentraut <peter_e@gmx.net> — 2009-11-12T15:28:57Z
On ons, 2009-11-04 at 16:02 +0200, Peter Eisentraut wrote: > Here is a patch to support arrays in PL/Python as parameters and > return values. Slightly updated version with fixed reference counting.
-
Re: PL/Python array support
Teodor Sigaev <teodor@sigaev.ru> — 2009-11-13T15:46:42Z
CREATE OR REPLACE FUNCTION incr(stuff int[]) RETURNS int[] AS $$ for x in stuff: yield x+1 $$ LANGUAGE 'plpythonu'; # select incr(ARRAY[1,2,3]); ERROR: invalid memory alloc request size 18446744073709551608 CONTEXT: while creating return value PL/Python function "incr" Suppose, it could be fixed by additional check in PLy_function_handler near line 947 : if (proc->is_setof) { ... } else if (PyIter_Check(plrv)) { ereport(ERROR, (errcode(ERRCODE_DATATYPE_MISMATCH), errmsg("returned object should be iterated"), errdetail("PL/Python returns iterable object in non-setof returning context"))); } Peter Eisentraut wrote: > On ons, 2009-11-04 at 16:02 +0200, Peter Eisentraut wrote: >> Here is a patch to support arrays in PL/Python as parameters and >> return values. > > Slightly updated version with fixed reference counting. > > > ------------------------------------------------------------------------ > > -- Teodor Sigaev E-mail: teodor@sigaev.ru WWW: http://www.sigaev.ru/ -
Re: PL/Python array support
Peter Eisentraut <peter_e@gmx.net> — 2009-11-19T22:00:24Z
On fre, 2009-11-13 at 18:46 +0300, Teodor Sigaev wrote: > CREATE OR REPLACE FUNCTION incr(stuff int[]) RETURNS int[] AS $$ > for x in stuff: > yield x+1 > $$ > LANGUAGE 'plpythonu'; > > # select incr(ARRAY[1,2,3]); > ERROR: invalid memory alloc request size 18446744073709551608 > CONTEXT: while creating return value > PL/Python function "incr" Fixed with additional error check and regression test. (The problem could be more simply demonstrated by returning any non-sequence from the function.) Thanks for catching it.
-
Re: PL/Python array support
Joshua Tolley <eggyknap@gmail.com> — 2009-12-02T03:53:49Z
On Fri, Nov 20, 2009 at 12:00:24AM +0200, Peter Eisentraut wrote: > On fre, 2009-11-13 at 18:46 +0300, Teodor Sigaev wrote: > > CREATE OR REPLACE FUNCTION incr(stuff int[]) RETURNS int[] AS $$ > > for x in stuff: > > yield x+1 > > $$ > > LANGUAGE 'plpythonu'; > > > > # select incr(ARRAY[1,2,3]); > > ERROR: invalid memory alloc request size 18446744073709551608 > > CONTEXT: while creating return value > > PL/Python function "incr" > > Fixed with additional error check and regression test. (The problem > could be more simply demonstrated by returning any non-sequence from the > function.) Thanks for catching it. I've finally gotten around to getting a review done, and basically it looks good to me. There appears to be a problem with the tests in that the expected output doesn't include the test_type_conversion_array_error() function mentioned in sql/plpython_types.sql. Diff generated by the regression test is attached. Other than that problem, though, the code looks fine to me (should I presume to judge Peter's code? :). Aside from the problem mentioned above, the tests work fine, and seem fairly comprehensive. Other testing I've done also passes. This patch doesn't include any documentation; my reading of the PL/Python docs suggests that's probably acceptable, as the existing docs don't talk about its array handling. That said, it might be useful to include an example, to show for instance that identical PL/Python code could create either an array of a type or a set of rows of that type: 5432 josh@josh*# create function return_set() returns setof int as $$ return (1, 2, 3, 4, 5) $$ language plpythonu; CREATE FUNCTION 5432 josh@josh*# create function return_arr() returns int[] as $$ return (1, 2, 3, 4, 5) $$ language plpythonu; CREATE FUNCTION 5432 josh@josh*# select return_arr(); return_arr ------------- {1,2,3,4,5} (1 row) 5432 josh@josh*# select * from return_set(); return_set ------------ 1 2 3 4 5 (5 rows) Perhaps that's overkill, though. -- Joshua Tolley / eggyknap End Point Corporation http://www.endpoint.com -
Re: PL/Python array support
Joshua Tolley <eggyknap@gmail.com> — 2009-12-02T12:11:56Z
On Fri, Nov 20, 2009 at 12:00:24AM +0200, Peter Eisentraut wrote: > On fre, 2009-11-13 at 18:46 +0300, Teodor Sigaev wrote: > > CREATE OR REPLACE FUNCTION incr(stuff int[]) RETURNS int[] AS $$ > > for x in stuff: > > yield x+1 > > $$ > > LANGUAGE 'plpythonu'; > > > > # select incr(ARRAY[1,2,3]); > > ERROR: invalid memory alloc request size 18446744073709551608 > > CONTEXT: while creating return value > > PL/Python function "incr" > > Fixed with additional error check and regression test. (The problem > could be more simply demonstrated by returning any non-sequence from the > function.) Thanks for catching it. My last email claimed that the regression test needed some additional changes to its expected output, and further claimed that it had the regression test's diff attached. As was helpfully pointed out off-list, it actually wasn't attached. Trying again.. -- Josh
-
Re: PL/Python array support
Peter Eisentraut <peter_e@gmx.net> — 2009-12-10T20:44:50Z
On tis, 2009-12-01 at 20:53 -0700, Joshua Tolley wrote: > This patch doesn't include any documentation; my reading of the PL/Python docs > suggests that's probably acceptable, as the existing docs don't talk about its > array handling. That said, it might be useful to include an example, to show > for instance that identical PL/Python code could create either an array of a > type or a set of rows of that type: I added a bit of documentation like that. Thanks.