Horton

Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at end-of-transaction

Justin Pryzby <pryzby@telsasoft.com>

View thread View on postgr.es
From: Justin Pryzby <pryzby@telsasoft.com>
To: Fabien COELHO <coelho@cri.ensmp.fr>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers@postgresql.org, Alvaro Herrera <alvherre@2ndquadrant.com>, David Steele <david@pgmasters.net>, "Bossart, Nathan" <bossartn@amazon.com>, Thomas Munro <thomas.munro@gmail.com>
Date: 2020-03-31T08:06:43Z
Lists: pgsql-hackers 
On Tue, Mar 31, 2020 at 07:36:03AM +0200, Fabien COELHO wrote:
> > > As I wrote about an earlier version of the patch, ISTM that instead of
> > > reinventing, extending, adapting various ls variants (with/without
> > > metadata, which show only files, which shows target of links, which shows
> > > directory, etc.) we would just need *one* postgres "ls" implementation
> > > which would be like "ls -la arg" (returns file type, dates), and then
> > > everything else is a wrapper around that with appropriate filtering that
> > > can be done at the SQL level, like you started with recurse.
> > 
> > Yeah, I agree that some new function that can represent symlinks
> > explicitly in its output is the place to deal with this, for
> > people who want to deal with it.
> > 
> > In the meantime, there's still the question of what pg_ls_dir_files
> > should do exactly.  Are we content to have it ignore symlinks?
> > I remain inclined to think that's the right thing given its current
> > brief.
> 
> My 0.02€:
> 
> I agree that it is enough to reproduce the current behavior of various
> existing pg_ls* functions, but on the other hand outputing a column type
> char like ls (-, d, l…) looks like really no big deal. I'd say that the only
> reason not to do it may be to pass this before feature freeze.

Remember, there's two threads here, and this one is about the bug in stable
releases ($SUBJECT), and now the instability in the test that was added with
its fix.

I suggest to leave stat() alone in your patch for stable releases.  I think
it's okay if we change behavior so that a broken symlink is skipped instead of
erroring (as a side effect of skipping ENOENT with stat()).  But not okay if we
change pg_ls_logdir() to hide symlinks in back braches.

-- 
Justin

Commits

  1. Teach pg_ls_dir_files() to ignore ENOENT failures from stat().

  2. Remove useless pfree()s at the ends of various ValuePerCall SRFs.

  3. Avoid holding a directory FD open across assorted SRF calls.

  4. Document pg_ls_*dir hiding of directories and special files

  5. Avoid holding a directory FD open across pg_ls_dir_files() calls.

  6. Adjust nodeFunctionscan.c to reset transient memory context between calls