Horton

Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at end-of-transaction

Fabien COELHO <coelho@cri.ensmp.fr>

View thread View on postgr.es
From: Fabien COELHO <coelho@cri.ensmp.fr>
To: Justin Pryzby <pryzby@telsasoft.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers@postgresql.org, Alvaro Herrera <alvherre@2ndquadrant.com>, David Steele <david@pgmasters.net>, "Bossart, Nathan" <bossartn@amazon.com>, Thomas Munro <thomas.munro@gmail.com>
Date: 2020-03-30T05:16:17Z
Lists: pgsql-hackers 
Hello Justin,

>> Well, the following comment says "ignore anything but regular files",
>> so I'm supposing that that is the behavior that we actually want here
>> and failed to implement correctly.  There might be scope for
>> additional directory-reading functions, but I'd think you'd want
>> more information (such as the file type) returned from anything
>> that doesn't act this way.
>
> Maybe pg_stat_file() deserves similar attention ?  Right now, it'll fail on a
> broken link.  If we changed it to lstat(), then it'd work, but it'd also show
> metadata for the *link* rather than its target.

Yep. I think this traditional answer is the rational answer.

As I wrote about an earlier version of the patch, ISTM that instead of 
reinventing, extending, adapting various ls variants (with/without 
metadata, which show only files, which shows target of links, which shows 
directory, etc.) we would just need *one* postgres "ls" implementation 
which would be like "ls -la arg" (returns file type, dates), and then 
everything else is a wrapper around that with appropriate filtering that 
can be done at the SQL level, like you started with recurse.

It would reduce the amount of C code and I find the SQL-level approach 
quite elegant.

-- 
Fabien.

Commits

  1. Teach pg_ls_dir_files() to ignore ENOENT failures from stat().

  2. Remove useless pfree()s at the ends of various ValuePerCall SRFs.

  3. Avoid holding a directory FD open across assorted SRF calls.

  4. Document pg_ls_*dir hiding of directories and special files

  5. Avoid holding a directory FD open across pg_ls_dir_files() calls.

  6. Adjust nodeFunctionscan.c to reset transient memory context between calls