Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT
Tom Lane <tgl@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
To: eric.cyr@gmail.com
Cc: pgsql-bugs@lists.postgresql.org, pgsql-hackers@lists.postgresql.org
Date: 2018-10-23T12:21:31Z
Lists: pgsql-bugs, pgsql-hackers
Attachments
- process-sigpipe-normally-in-copy-from-program-1.patch (text/x-patch) patch
I wrote: > =?utf-8?q?PG_Bug_reporting_form?= <noreply@postgresql.org> writes: >> SELECT * FROM test_file_fdw_program_limit LIMIT 0; >> /* >> [38000] ERROR: program "echo "test"" failed Detail: child process exited >> with exit code 1 >> */ > Yeah, I can reproduce this on macOS as well as Linux. Capturing stderr > shows something pretty unsurprising: > sh: line 1: echo: write error: Broken pipe > So the called program is behaving in a somewhat reasonable way: it's > detecting EPIPE on its stdout (after we close the pipe), reporting that, > and doing exit(1). > Unfortunately, it's not clear what we could do about that, short of > always reading the whole program output, which is likely to be a > cure worse than the disease. If the program were failing thanks to > SIGPIPE, we could recognize that as a case we can ignore ... but with > behavior like this, I don't see a reliable way to tell it apart from > a generic program failure, which surely we'd better report. After a bit of thought, the problem here is blindingly obvious: we generally run the backend with SIGPIPE handing set to SIG_IGN, and evidently popen() allows the called program to inherit that, at least on these platforms. So what we need to do is make sure the called program inherits SIG_DFL handling for SIGPIPE, and then special-case that result as not being a failure. The attached POC patch does that and successfully makes the file_fdw problem go away for me. It's just a POC because there are some things that need more thought than I've given them: 1. Is it OK to revert SIGPIPE to default processing for *all* programs launched through OpenPipeStream? If not, what infrastructure do we need to add to control that? In particular, is it sane to revert SIGPIPE for a pipe program that we will write to not read from? (I thought probably yes, because that is the normal Unix behavior, but it could be debated.) 2. Likewise, is it always OK for ClosePipeToProgram to ignore a SIGPIPE failure? (For ordinary COPY, maybe it shouldn't, since we don't intend to terminate that early.) 3. Maybe this should be implemented at some higher level? 4. Are there any other signals we ought to be reverting to default behavior before launching a COPY TO/FROM PROGRAM? regards, tom lane
Commits
-
Handle EPIPE more sanely when we close a pipe reading from a program.
- ffa4cbd623dd 12.0 landed
- 8dc49a8934de 11.2 landed
- 8285fae0706f 10.7 landed
-
Avoid defining SIGTTIN/SIGTTOU on Windows.
- 37afc079abe1 12.0 landed
-
Leave SIGTTIN/SIGTTOU signal handling alone in postmaster child processes.
- 125f551c8be1 12.0 landed
-
Remove some unnecessary pqsignal() calls to shave a few cycles off
- 8e2998d8a6ae 7.3.1 cited
-
Fix handling of SIGCHLD, per recent pghackers discussion: on some
- dad8e410d0c4 7.2.1 cited
-
The new files for statistical system views.
- 140ddb78fe62 7.2.1 cited
-
Remove fork()/exec() and only do fork(). Small cleanups.
- 212c905e2c5a 7.1.1 cited