Re: PATCH: pgbench - option to build using ppoll() for larger connection counts
Tom Lane <tgl@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Fabien COELHO <coelho@cri.ensmp.fr>
Cc: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>,
Thomas Munro <thomas.munro@enterprisedb.com>,
"Rady,
Doug" <radydoug@amazon.com>,
Andres Freund <andres@anarazel.de>,
"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>,
Robert Haas <robertmhaas@gmail.com>
Date: 2018-09-23T00:11:34Z
Lists: pgsql-hackers
Attachments
- pgbench-ppoll-17.patch (text/x-diff) patch
I wrote: > I'm strongly tempted to just remove the POLL_UNWANTED business > altogether, as it seems both pointless and unportable on its face. > Almost by definition, we can't know what "other" bits a given > implementation might set. > I'm not entirely following the point of including POLLRDHUP in > POLL_EVENTS, either. What's wrong with the traditional solution > of detecting EOF? So after studying that a bit longer, I think it's just wrong. It's not the business of this code to be checking for connection errors at all; that is libpq's province. The libpq API specifies that callers should wait for read-ready on the socket, and nothing else. So the only bit we need concern ourselves with is POLLIN. I also seriously disliked both the details of the abstraction API and its lack of documentation. (Other people complained about that upthread, too.) So attached is a rewrite attempt. There's still a couple of grotty things about it; in particular the ppoll variant of socket_has_input() knows more than one could wish about how it's being used. But I couldn't see a way to make it cleaner without significant changes to the logic in threadRun, and that didn't seem better. I think that Andres' concern upthread about iterating over a whole lot of sockets is somewhat misplaced. We aren't going to be iterating over the entire set of client connections, only those being run by a particular pgbench thread. So assuming you're using a reasonable ratio of threads to clients, there won't be very many to look at in any one thread. In any case, I'm dubious that we could get much of a win from some other abstraction for waiting: both of these code paths do work pretty much proportional to the number of connections the current thread is responsible for, and it's hard to see how to avoid that. I've tested this on both Linux and FreeBSD, and it seems to work fine. I'm reasonably happy with this version of the patch, and would be ready to commit it, but I thought I'd throw it out for another round of review if anyone wants to. regards, tom lane
Commits
-
Use ppoll(2), if available, to wait for input in pgbench.
- 60e612b60299 12.0 landed
-
Use pselect(2) not select(2), if available, to wait in postmaster's loop.
- 81069a9efc5a 10.0 cited