Re: Review: Extra Daemons / bgworker

Alvaro Herrera <alvherre@2ndquadrant.com>

From: Alvaro Herrera <alvherre@2ndquadrant.com>
To: Markus Wanner <markus@bluegap.ch>
Cc: kaigai@kaigai.gr.jp, PostgreSQL-development Hackers <pgsql-hackers@postgresql.org>
Date: 2012-12-03T21:35:44Z
Lists: pgsql-hackers

Attachments

So here's version 8.  This fixes a couple of bugs and most notably
creates a separate PGPROC list for bgworkers, so that they don't
interfere with client-connected backends.

I tested starting 1000 non-shmem attached bgworkers -- postmaster
doesn't break a sweat.  Also running 200 backend-connected bgworkers
works fine (assuming they do nothing).  This is on my laptop, which is a
dual-core Intel i5 processor.

One notable thing is that I had to introduce this in the postmaster
startup sequence:

    /*
     * process any libraries that should be preloaded at postmaster start
     */
    process_shared_preload_libraries();

    /*
     * If loadable modules have added background workers, MaxBackends needs to
     * be updated.  Do so now.
     */
    // RerunAssignHook("max_connections");
    if (GetNumShmemAttachedBgworkers() > 0)
        SetConfigOption("max_connections",
                        GetConfigOption("max_connections", false, false),
                        PGC_POSTMASTER, PGC_S_OVERRIDE);

Note the intention here is to re-run the GUC assign hook for
max_connections (hence the commented out hypothetical call to do so).
Obviously, having to go through GetConfigOption and SetConfigOption is
not a nice thing to do; we'll have to add some new entry point to guc.c
for this to have a nicer interface.

(I also observed that it's probably a good idea to have something like
FunctionSetConfigOption for the places that are currently calling
set_config_option directly).

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services