Re: exposing COPY API

Andrew Dunstan <andrew@dunslane.net>

From: Andrew Dunstan <andrew@dunslane.net>
To: Itagaki Takahiro <itagaki.takahiro@gmail.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2011-02-04T02:32:31Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Allow the low level COPY routines to read arbitrary numbers of fields.


>> Of course, I want this so I could construct a text array from the read in
>> data, but I could also imagine a foreign data wrapper wanting to mangle the
>> data before handing it to postgres, say by filling in a field or hashing it.
> Could you explain the actual use-cases and examples?  I think we need to have
> SQL-level extensibility if we provide such flexibility. I guess typical users
> don't want to write functions with C for each kind of input files.
>
> Note that pg_bulkload has a similar feature like as:
>    CREATE FUNCTION my_function(...) RETURNS record AS ...;
>    COPY tbl FROM 'file' WITH (make_record_from_line = my_function)


Umm, where? I can't find this in the documentation 
<http://pgbulkload.projects.postgresql.org/pg_bulkload.html> nor in the 
source code. And how would  module like that provide an extra copy option?

The object, as I have explained previously, is to have a FDW that 
returns a text array from a (possibly irregularly shaped) file.


So, given this file:

1,,2,3
4,5,6

select t[4] as a,t[2] as b from my_fdw_table;

would return

a | b
-----
3 |
   | 5



cheers

andrew