Horton

Thread

Commits

  1. Use setproctitle_fast() to update the ps status, if available.

  1. setproctitle_fast()

    Thomas Munro <thomas.munro@enterprisedb.com> — 2018-07-04T22:09:09Z

    Hello hackers,

FreeBSD's setproctitle() is a bit slow because it contains a syscall
or two, so people often run PostgreSQL with update_process_title set
to off on that OS.  That makes the user experience not quite as nice
as Linux.  As a weekend learn-me-some-kernel-hacking project I fixed
that and got the patch committed to FreeBSD 12, though I was asked to
use a new libc entry point _fast().  Here's a patch to teach
PostgreSQL about that.  It doesn't have much effect on small systems,
but it makes "pgbench -c 40 -j 40 -S -M prepared" do ~10% more
transactions per second on an AWS m4.10xlarge instance.

I'll park this patch here until the FreeBSD feature escapes in a
RELEASE version in a few months.

For anyone interested, https://wiki.postgresql.org/wiki/FreeBSD has
some notes on this and other such things.

-- 
Thomas Munro
http://www.enterprisedb.com

  2. Re: setproctitle_fast()

    Emre Hasegeli <emre@hasegeli.com> — 2018-07-07T11:57:01Z

    > FreeBSD's setproctitle() is a bit slow because it contains a syscall
> or two, so people often run PostgreSQL with update_process_title set
> to off on that OS.  That makes the user experience not quite as nice
> as Linux.  As a weekend learn-me-some-kernel-hacking project I fixed
> that and got the patch committed to FreeBSD 12, though I was asked to
> use a new libc entry point _fast().  Here's a patch to teach
> PostgreSQL about that.  It doesn't have much effect on small systems,
> but it makes "pgbench -c 40 -j 40 -S -M prepared" do ~10% more
> transactions per second on an AWS m4.10xlarge instance.

I am curious why they asked you to use a new libc entry point.  The
function signatures are the same.  More people could benefit from
making the existing function faster.

  3. Re: setproctitle_fast()

    Thomas Munro <thomas.munro@enterprisedb.com> — 2018-07-07T13:11:11Z

    On Sat, Jul 7, 2018 at 11:57 PM, Emre Hasegeli <emre@hasegeli.com> wrote:
>> FreeBSD's setproctitle() is a bit slow because it contains a syscall
>> or two, so people often run PostgreSQL with update_process_title set
>> to off on that OS.  That makes the user experience not quite as nice
>> as Linux.  As a weekend learn-me-some-kernel-hacking project I fixed
>> that and got the patch committed to FreeBSD 12, though I was asked to
>> use a new libc entry point _fast().  Here's a patch to teach
>> PostgreSQL about that.  It doesn't have much effect on small systems,
>> but it makes "pgbench -c 40 -j 40 -S -M prepared" do ~10% more
>> transactions per second on an AWS m4.10xlarge instance.
>
> I am curious why they asked you to use a new libc entry point.  The
> function signatures are the same.  More people could benefit from
> making the existing function faster.

There is a small trade-off: ps/top/htop do a bit more work (the kernel
has to go and pull the data out of the source process's memory instead
of keeping its own copy maintained via syscalls).  Originally I
proposed to make setproctitle() adaptive, figuring out who were the
frequent updaters, but the heuristics I proposed were rejected and it
looked hard to come up with better ones[1].  There may also be an
argument that you should have to opt into the fast and loose
consistency of this approach explicitly (though no one seems to mind
about that on Linux).

Anyway, I guess there aren't too many other applications that want to
do this kind of thing at 1MHz or whatever!

[1] https://lists.freebsd.org/pipermail/freebsd-hackers/2018-July/052975.html

-- 
Thomas Munro
http://www.enterprisedb.com

  4. Re: setproctitle_fast()

    Thomas Munro <thomas.munro@enterprisedb.com> — 2018-07-24T01:11:43Z

    On Thu, Jul 5, 2018 at 10:09 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
> I'll park this patch here until the FreeBSD feature escapes in a
> RELEASE version in a few months.

It has now been merged into FreeBSD stable/11, so it's expected in
FreeBSD 11.3.  It is supported by buildfarm animal peripatus.  So I
figured I might as well push this to master now, to increase the time
it gets tested on both sides.  Done.

I won't back-patch it, per project policy, but I've proposed local
patches for FreeBSD's ports over here:
https://reviews.freebsd.org/D16234

-- 
Thomas Munro
http://www.enterprisedb.com