Re: pgsql: Introduce pg_shmem_allocations_numa view

Tomas Vondra <tomas@vondra.me>

From: Tomas Vondra <tomas@vondra.me>

To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>, Christoph Berg <myon@debian.org>

Cc: Andres Freund <andres@anarazel.de>, Tomas Vondra <tomas.vondra@postgresql.org>, pgsql-hackers@lists.postgresql.org

Date: 2025-06-27T14:52:08Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Handle EPERM in pg_numa_init
- 599336c64fc9 19 (unreleased) landed
- 482e98ac4302 18.2 landed
Add CHECK_FOR_INTERRUPTS into pg_numa_query_pages
- 54ac4944c36f 18.0 landed
- bf1119d74a79 19 (unreleased) landed
Silence valgrind about pg_numa_touch_mem_if_required
- 14e52227e578 18.0 landed
- 81f287dc923f 19 (unreleased) landed
Limit the size of numa_move_pages requests
- 45879f48f140 18.0 landed
- 7fe2f67c7c9f 19 (unreleased) landed
Introduce pg_shmem_allocations_numa view
- 8cc139bec34a 18.0 cited

Attachments

0001-Add-batching-when-calling-numa_move_pages.patch (text/x-patch) patch 0001
0002-Silence-valgrind-about-pg_numa_touch_mem_if_required.patch (text/x-patch) patch 0002
0003-Add-CHECK_FOR_INTERRUPTS-into-pg_numa_query_pages.patch (text/x-patch) patch 0003

Here's three small patches, that should handle the issue


0001 - Adds the batching into pg_numa_query_pages, so that the callers
don't need to do anything.

The batching doesn't seem to cause any performance regression. 32-bit
systems can't use that much memory anyway, and on 64-bit systems the
batch is sufficiently large (1024).


0002 - Silences the valgrind about the memory touching. It replaces the
macro with a static inline function, and adds suppressions for both
32-bit and 64-bits. The 32-bit may be a bit pointless, because on my
rpi5 valgrind produces about a bunch of other stuff anyway. But doesn't
hurt.

The function now looks like this:

  static inline void
  pg_numa_touch_mem_if_required(void *ptr)
  {
      volatile uint64 touch pg_attribute_unused();
      touch = *(volatile uint64 *) ptr;
  }

I did a lot of testing on multiple systems to check replacing the macro
with a static inline function still works - and it seems it does. But if
someone thinks the function won't work, I'd like to know.


0003 - While working on these patches, it occurred to me we could/should
add CHECK_FOR_INTERRUPTS() into the batch loop. This querying can take
quite a bit of time, so letting people to interrupt it seems reasonable.
It wasn't possible with just one call into the kernel, but with the
batching we can add a CFI.


Please, take a look.

regards

-- 
Tomas Vondra