Re: textarray option for file FDW

On 01/15/2011 07:41 PM, Andrew Dunstan wrote:
>
>
> On 01/15/2011 12:29 PM, Andrew Dunstan wrote:
>>
>> I've been waiting for the latest FDW patches as patiently as I can, 
>> and I've been reviewing them this morning, in particular the file_fdw 
>> patch and how it interacts with the newly exposed COPY API. Overall 
>> it seems to be a whole lot cleaner, and the wholesale duplication of 
>> the copy code is gone, so it's much nicer and cleaner. So now I'd 
>> like to add a new option to it: "textarray". This option would 
>> require that the foreign table have exactly one field, of type 
>> text[], and would compose all the field strings read from the file 
>> for each record into the array (however many there are). This would 
>> require a few changes to contrib/file_fdw/file_fdw.c and a few 
>> changes to src/backend/commands/copy.c, which I can probably have 
>> done in fairly short order, Deo Volente. This will allow something like:
>>
>>    CREATE FOREIGN TABLE arr_text (
>>         t text[]
>>    )  SERVER file_server
>>    OPTIONS (format 'csv', filename '/path/to/ragged.csv', textarray
>>    'true');
>>    SELECT t[3]::int as a, t[1]::timestamptz as b, t[99] as not_there
>>    FROM arr_text;
>>
>>
>
> A WIP patch is attached. It's against Shigeru Hanada's latest FDW 
> patches. It's surprisingly tiny. Right now it probably leaks memory 
> like a sieve, and that's the next thing I'm going to chase down.
>
>

Updated patch attached, that should use memory better.

cheers

andrew