Re: allow changing autovacuum_max_workers without restarting

Sami Imseih <simseih@amazon.com>

From: "Imseih (AWS), Sami" <simseih@amazon.com>
To: Nathan Bossart <nathandbossart@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2024-04-11T14:24:18Z
Lists: pgsql-hackers
> I frequently hear about scenarios where users with thousands upon thousands
> of tables realize that autovacuum is struggling to keep up.  When they
> inevitably go to bump up autovacuum_max_workers, they discover that it
> requires a server restart (i.e., downtime) to take effect, causing further
> frustration.  For this reason, I think $SUBJECT is a desirable improvement.
> I spent some time looking for past discussions about this, and I was
> surprised to not find any, so I thought I'd give it a try.

I did not review the patch in detail yet, but +1 to the idea. 
It's not just thousands of tables that suffer from this.
If a user has a few large tables hogging the autovac workers, then other
tables don't get the autovac cycles they require. Users are then forced
to run manual vacuums, which adds complexity to their operations.

> The attached proof-of-concept patch demonstrates what I have in mind.
> Instead of trying to dynamically change the global process table, etc., I'm
> proposing that we introduce a new GUC that sets the effective maximum
> number of autovacuum workers that can be started at any time.

max_worker_processes defines a pool of max # of background workers allowed.
parallel workers and extensions that spin  up background workers all utilize from 
this pool. 

Should autovacuum_max_workers be able to utilize from max_worker_processes also?

This will allow autovacuum_max_workers to be dynamic while the user only has
to deal with an already existing GUC. We may want to increase the default value
for max_worker_processes as part of this.


Regards,

Sami
Amazon Web Services (AWS)

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Give up on running with NetBSD/OpenBSD's default semaphore settings.

  2. initdb: Do not report default autovacuum_worker_slots.

  3. Lower default value of autovacuum_worker_slots in initdb as needed.

  4. Allow changing autovacuum_max_workers without restarting.

  5. Reserve a PGPROC slot and semaphore for the slotsync worker process.

  6. Make archiver process an auxiliary process.

  7. Correct the formulas for System V IPC parameters SEMMNI and SEMMNS in docs.