Horton

Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at end-of-transaction

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Justin Pryzby <pryzby@telsasoft.com>
Cc: pgsql-hackers@postgresql.org, Fabien COELHO <coelho@cri.ensmp.fr>, Alvaro Herrera <alvherre@2ndquadrant.com>, David Steele <david@pgmasters.net>, "Bossart, Nathan" <bossartn@amazon.com>, Thomas Munro <thomas.munro@gmail.com>
Date: 2020-03-11T19:32:38Z
Lists: pgsql-hackers 
Justin Pryzby <pryzby@telsasoft.com> writes:
>>> On Sun, Mar 08, 2020 at 02:37:49PM -0400, Tom Lane wrote:
>>>> I guess we ought to change that function to use returns-a-tuplestore
>>>> protocol instead of thinking it can hold a directory open across calls.

> I patched this one to see what it looks like and to allow /hopefully/ moving
> forward one way or another with the pg_ls_tmpfile() patch set (or at least
> avoid trying to do anything there which is too inconsistent with this fix).

I reviewed this, added some test cases, and pushed it, so that we can see
if the buildfarm finds anything wrong.  (I'm not expecting that, because
this should all be pretty portable, but you never know.)  Assuming not,
we need to fix the other functions similarly, and then do something about
revising the documentation to warn against this coding style.  Do you
want to have a go at that?

> Also, at first glance, these seem to be passing constant "randomAccess=true"
> rather than (bool) (rsinfo->allowedModes&SFRM_Materialize_Random)

Hm.  Not a bug, but possibly a performance issue, if the tuplestore
gets big enough for that to matter.  (I think it doesn't matter until
we start spilling to temp files.)

			regards, tom lane

Commits

  1. Teach pg_ls_dir_files() to ignore ENOENT failures from stat().

  2. Remove useless pfree()s at the ends of various ValuePerCall SRFs.

  3. Avoid holding a directory FD open across assorted SRF calls.

  4. Document pg_ls_*dir hiding of directories and special files

  5. Avoid holding a directory FD open across pg_ls_dir_files() calls.

  6. Adjust nodeFunctionscan.c to reset transient memory context between calls