Horton
Menu

Re: Automatically sizing the IO worker pool

Thomas Munro <thomas.munro@gmail.com>

View thread
From: Thomas Munro <thomas.munro@gmail.com>
To: Dmitry Dolgov <9erthalion6@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2026-04-11T06:35:18Z
Lists: pgsql-hackers

Attachments

On Wed, Jul 30, 2025 at 10:15 PM Dmitry Dolgov <9erthalion6@gmail.com> wrote:
> As a side note, I was trying to experiment with this patch using
> dm-mapper's delay feature to introduce an arbitrary large io latency and
> see how the io queue is growing.

FWIW, here's what I came up with while experimenting with that sort of thing:

      shared_preload_libraries=io_limit
      io_limit.ios_per_second=6000

That differs from eg dm-mapper delays by making everything seem like
slow direct I/O, which seemed more interesting for this project.  For
example if you run some continuous workload while you SET
io_limit.ios_per_second to various numbers, with
io_workers_idle_timeout set fairly low, you can monitor the pool
adjustments.

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. aio: Adjust I/O worker pool automatically.

  2. Convert lwlock.c to use the new shmem allocation functions

  3. aio: Simplify pgaio_worker_submit().

  4. Conditional locking in pgaio_worker_submit_internal

  5. aio: Remove obsolete IO worker ID references.

  6. aio: Regularize IO worker internal naming.