Re: scalability bottlenecks with (many) partitions (and more)

Tomas Vondra <tomas@vondra.me>

From: Tomas Vondra <tomas@vondra.me>

To: Robert Haas <robertmhaas@gmail.com>

Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Andres Freund <andres@anarazel.de>

Date: 2024-09-12T23:44:58Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Make FP_LOCK_SLOTS_PER_BACKEND look like a function
- c878de1db438 18.0 landed
Fix asserts in fast-path locking code
- a7e5237f268e 18.0 landed
Increase the number of fast-path lock slots
- c4d5cb71d229 18.0 landed

Attachments

v20240913-0001-Increase-the-number-of-fast-path-lock-slot.patch (text/x-patch) patch v20240913-0001
v20240913-0002-Set-fast-path-slots-using-max_locks_per_tr.patch (text/x-patch) patch v20240913-0002

Turns out there was a bug in EXEC_BACKEND mode, causing failures on the
Windows machine in CI. AFAIK the reason is pretty simple - the backends
don't see the number of fast-path groups postmaster calculated from
max_locks_per_transaction.

Fixed that by calculating it again in AttachSharedMemoryStructs, which
seems to have done the trick. With this the CI builds pass just fine,
but I'm not sure if EXEC_BACKENDS may have some other issues with the
PGPROC changes. Could it happen that the shared memory gets mapped
differently, in which case the pointers might need to be adjusted?


regards

-- 
Tomas Vondra