Horton

Thread

Commits

  1. Change path in example of file_fdw for logs

  2. Fix docs bug stating file_fdw requires absolute paths

  3. Add contrib/file_fdw foreign-data wrapper for reading files via COPY.

  1. file_fdw vs relative paths

    Magnus Hagander <magnus@hagander.net> — 2020-07-15T11:22:21Z

    According to the documentation, the filename given in file_fdw must be an
absolute path. Hwever, it works perfectly fine with a relative path.

So either the documentation is wrong, or the code is wrong. It behaves the
same at least back to 9.5, I did not try it further back than that.

I can't find a reference to the code that limits this. AFAICT the
documentation has been there since day 1.

Question is, which one is right. Is there a reason we'd want to restrict it
to absolute pathnames?

-- 
 Magnus Hagander
 Me: https://www.hagander.net/ <http://www.hagander.net/>
 Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>

  2. Re: file_fdw vs relative paths

    Bruce Momjian <bruce@momjian.us> — 2020-08-25T00:26:12Z

    On Wed, Jul 15, 2020 at 01:22:21PM +0200, Magnus Hagander wrote:
> According to the documentation, the filename given in file_fdw must be an
> absolute path. Hwever, it works perfectly fine with a relative path.
> 
> So either the documentation is wrong, or the code is wrong. It behaves the same
> at least back to 9.5, I did not try it further back than that.

Yes, I tested back to 9.5 too:

	CREATE EXTENSION file_fdw;
	CREATE SERVER pgconf FOREIGN DATA WRAPPER file_fdw;
	CREATE FOREIGN TABLE pgconf (line TEXT) SERVER pgconf OPTIONS ( filename
		'postgresql.conf', format 'text', delimiter E'\x7f' );
	SELECT * FROM pgconf;
	 # -----------------------------
	 # PostgreSQL configuration file
	 # -----------------------------
	 #
	 # This file consists of lines of the form:
	...

> I can't find a reference to the code that limits this. AFAICT the documentation
> has been there since day 1.
> 
> Question is, which one is right. Is there a reason we'd want to restrict it to
> absolute pathnames?

I think it should work just like COPY, which allows relative paths;  doc
patch attached.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee

  3. Re: file_fdw vs relative paths

    Japin Li <japinli@hotmail.com> — 2020-08-25T07:28:41Z

    On Aug 25, 2020, at 8:26 AM, Bruce Momjian <bruce@momjian.us<mailto:bruce@momjian.us>> wrote:

Yes, I tested back to 9.5 too:

CREATE EXTENSION file_fdw;
CREATE SERVER pgconf FOREIGN DATA WRAPPER file_fdw;
CREATE FOREIGN TABLE pgconf (line TEXT) SERVER pgconf OPTIONS ( filename
'postgresql.conf', format 'text', delimiter E'\x7f' );
SELECT * FROM pgconf;
 # -----------------------------
 # PostgreSQL configuration file
 # -----------------------------
 #
 # This file consists of lines of the form:
…

The file_fdw extension was introduced by commit 7c5d0ae7078456bfeedb2103c45b9a32285c2631,
and I tested it supports relative paths.  This is a doc bug.

--
Japin Li

  4. Re: file_fdw vs relative paths

    Magnus Hagander <magnus@hagander.net> — 2020-08-31T11:10:58Z

    On Tue, Aug 25, 2020 at 9:28 AM Li Japin <japinli@hotmail.com> wrote:

>
> On Aug 25, 2020, at 8:26 AM, Bruce Momjian <bruce@momjian.us> wrote:
>
> Yes, I tested back to 9.5 too:
>
> CREATE EXTENSION file_fdw;
> CREATE SERVER pgconf FOREIGN DATA WRAPPER file_fdw;
> CREATE FOREIGN TABLE pgconf (line TEXT) SERVER pgconf OPTIONS ( filename
> 'postgresql.conf', format 'text', delimiter E'\x7f' );
> SELECT * FROM pgconf;
>  # -----------------------------
>  # PostgreSQL configuration file
>  # -----------------------------
>  #
>  # This file consists of lines of the form:
> …
>
>
> The file_fdw extension was introduced by
> commit 7c5d0ae7078456bfeedb2103c45b9a32285c2631,
> and I tested it supports relative paths.  This is a doc bug.
>
>
Well technically it can also have been a code bug but yes if so it is one
that has lived since day 1. But given that nobody has chimed in to say they
think that's what it is for a month, I think we'll conclude it's a docs
bug.

Bruce, I've applied and backpatched your docs patch for this.

-- 
 Magnus Hagander
 Me: https://www.hagander.net/ <http://www.hagander.net/>
 Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>

  5. Re: file_fdw vs relative paths

    Magnus Hagander <magnus@hagander.net> — 2020-08-31T11:16:05Z

    On Mon, Aug 31, 2020 at 1:10 PM Magnus Hagander <magnus@hagander.net> wrote:

>
>
> On Tue, Aug 25, 2020 at 9:28 AM Li Japin <japinli@hotmail.com> wrote:
>
>>
>> On Aug 25, 2020, at 8:26 AM, Bruce Momjian <bruce@momjian.us> wrote:
>>
>> Yes, I tested back to 9.5 too:
>>
>> CREATE EXTENSION file_fdw;
>> CREATE SERVER pgconf FOREIGN DATA WRAPPER file_fdw;
>> CREATE FOREIGN TABLE pgconf (line TEXT) SERVER pgconf OPTIONS ( filename
>> 'postgresql.conf', format 'text', delimiter E'\x7f' );
>> SELECT * FROM pgconf;
>>  # -----------------------------
>>  # PostgreSQL configuration file
>>  # -----------------------------
>>  #
>>  # This file consists of lines of the form:
>> …
>>
>>
>> The file_fdw extension was introduced by
>> commit 7c5d0ae7078456bfeedb2103c45b9a32285c2631,
>> and I tested it supports relative paths.  This is a doc bug.
>>
>>
> Well technically it can also have been a code bug but yes if so it is one
> that has lived since day 1. But given that nobody has chimed in to say they
> think that's what it is for a month, I think we'll conclude it's a docs
> bug.
>
> Bruce, I've applied and backpatched your docs patch for this.
>
>
Gah, and of course right after doing that, I remembered I wanted to get a
second change in :) To solve the "who's this Josh" question, I suggest we
also change the example to point to the data/log directory which is likely
to exist in a lot more of the cases. I keep getting people who ask "who is
josh" based on the /home/josh path. Not that it's that important, but...

-- 
 Magnus Hagander
 Me: https://www.hagander.net/ <http://www.hagander.net/>
 Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>

  6. Re: file_fdw vs relative paths

    Bruce Momjian <bruce@momjian.us> — 2020-08-31T15:03:45Z

    On Mon, Aug 31, 2020 at 01:16:05PM +0200, Magnus Hagander wrote:
>     Bruce, I've applied and backpatched your docs patch for this.
> 
> Gah, and of course right after doing that, I remembered I wanted to get a
> second change in :) To solve the "who's this Josh" question, I suggest we also
> change the example to point to the data/log directory which is likely to exist
> in a lot more of the cases. I keep getting people who ask "who is josh" based
> on the /home/josh path. Not that it's that important, but... 

Thanks, and agreed.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee

  7. Re: file_fdw vs relative paths

    Magnus Hagander <magnus@hagander.net> — 2020-09-06T17:31:08Z

    On Mon, Aug 31, 2020 at 5:03 PM Bruce Momjian <bruce@momjian.us> wrote:

> On Mon, Aug 31, 2020 at 01:16:05PM +0200, Magnus Hagander wrote:
> >     Bruce, I've applied and backpatched your docs patch for this.
> >
> > Gah, and of course right after doing that, I remembered I wanted to get a
> > second change in :) To solve the "who's this Josh" question, I suggest
> we also
> > change the example to point to the data/log directory which is likely to
> exist
> > in a lot more of the cases. I keep getting people who ask "who is josh"
> based
> > on the /home/josh path. Not that it's that important, but...
>
> Thanks, and agreed.
>
>
Thanks, applied. I backpacked to 13 but didn't bother with the rest as it's
not technically *wrong* before..

//Magnus

  8. Re: file_fdw vs relative paths

    Ian Barwick <ian.barwick@2ndquadrant.com> — 2020-09-09T01:39:06Z

    Hi

On 2020/09/07 2:31, Magnus Hagander wrote:
> On Mon, Aug 31, 2020 at 5:03 PM Bruce Momjian <bruce@momjian.us <mailto:bruce@momjian.us>> wrote:
> 
>     On Mon, Aug 31, 2020 at 01:16:05PM +0200, Magnus Hagander wrote:
>      >     Bruce, I've applied and backpatched your docs patch for this.
>      >
>      > Gah, and of course right after doing that, I remembered I wanted to get a
>      > second change in :) To solve the "who's this Josh" question, I suggest we also
>      > change the example to point to the data/log directory which is likely to exist
>      > in a lot more of the cases. I keep getting people who ask "who is josh" based
>      > on the /home/josh path. Not that it's that important, but...
> 
>     Thanks, and agreed.
> 
> 
> Thanks, applied. I backpacked to 13 but didn't bother with the rest as it's not technically *wrong* before..

It's missing the leading single quote from the filename parameter:

     diff --git a/doc/src/sgml/file-fdw.sgml b/doc/src/sgml/file-fdw.sgml
     (...)
     -OPTIONS ( filename '/home/josh/data/log/pglog.csv', format 'csv' );
     +OPTIONS ( filename log/pglog.csv', format 'csv' );
     (...)


Regards


Ian Barwick


-- 
Ian Barwick                   https://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services

  9. Re: file_fdw vs relative paths

    Magnus Hagander <magnus@hagander.net> — 2020-09-09T10:42:51Z

    On Wed, Sep 9, 2020 at 3:39 AM Ian Barwick <ian.barwick@2ndquadrant.com>
wrote:

> Hi
>
> On 2020/09/07 2:31, Magnus Hagander wrote:
> > On Mon, Aug 31, 2020 at 5:03 PM Bruce Momjian <bruce@momjian.us <mailto:
> bruce@momjian.us>> wrote:
> >
> >     On Mon, Aug 31, 2020 at 01:16:05PM +0200, Magnus Hagander wrote:
> >      >     Bruce, I've applied and backpatched your docs patch for this.
> >      >
> >      > Gah, and of course right after doing that, I remembered I wanted
> to get a
> >      > second change in :) To solve the "who's this Josh" question, I
> suggest we also
> >      > change the example to point to the data/log directory which is
> likely to exist
> >      > in a lot more of the cases. I keep getting people who ask "who is
> josh" based
> >      > on the /home/josh path. Not that it's that important, but...
> >
> >     Thanks, and agreed.
> >
> >
> > Thanks, applied. I backpacked to 13 but didn't bother with the rest as
> it's not technically *wrong* before..
>
> It's missing the leading single quote from the filename parameter:
>
>      diff --git a/doc/src/sgml/file-fdw.sgml b/doc/src/sgml/file-fdw.sgml
>      (...)
>      -OPTIONS ( filename '/home/josh/data/log/pglog.csv', format 'csv' );
>      +OPTIONS ( filename log/pglog.csv', format 'csv' );
>      (...)
>

GAH.

Thanks!


-- 
 Magnus Hagander
 Me: https://www.hagander.net/ <http://www.hagander.net/>
 Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>