Thread

bg worker: patch 2 of 6 - job cache

Markus Wanner <markus@bluegap.ch> — 2010-07-13T14:31:07Z

This is the major refactoring patch, which turns the background workers 
into more permanent work-horses. After having processed a job, they now 
stay connected to their database and wait for more jobs from the 
coordinator.

On the coordinator side, a job cache got added, so it is able to queue 
requests for jobs on databases which don't currently have an idle 
background worker available. The existing autovacuum functionality got 
separated from the background worker infrastructure.

The coordinator uses a transaction to read pg_database and start 
background workers as appropriate. However, autovacuum still uses 
pg_stat, which needs to get initialized after startup. The coordinator 
triggers an initial VACUUM on the template database (template1), which 
populates the required statistics to kick off autovacuum.

Note that none of the statistics functions differenciate between 
autovacuum workers and other background job types, which might or might 
not be what we want.

Another open issue is the starvation problem: jobs for a database, which 
doesn't currently have any connected background workers might starve, if 
the coordinator isn't allowed to fork any new worker (and all others are 
idle on other databases).