Re: Draft for basic NUMA observability

Andres Freund <andres@anarazel.de>

From: Andres Freund <andres@anarazel.de>
To: Tomas Vondra <tomas@vondra.me>
Cc: Kirill Reshke <reshkekirill@gmail.com>, Jakub Wartak <jakub.wartak@enterprisedb.com>, Bertrand Drouvot <bertranddrouvot.pg@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Nazir Bilal Yavuz <byavuz81@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2025-04-08T23:29:08Z
Lists: pgsql-hackers
Hi,

On 2025-04-09 01:10:09 +0200, Tomas Vondra wrote:
> On 4/8/25 15:06, Andres Freund wrote:
> > Hi,
> > 
> > On 2025-04-08 17:44:19 +0500, Kirill Reshke wrote:
> >> On Mon, 7 Apr 2025 at 23:00, Tomas Vondra <tomas@vondra.me> wrote:
> >>> I'll let the CI run the tests on it, and
> >>> then will push, unless someone has more comments.
> >>>
> >>
> >>
> >> Hi! I noticed strange failure after this commit[0]
> >>
> >> Looks like it is related to 65c298f61fc70f2f960437c05649f71b862e2c48
> >>
> >> In file included from  [01m [K../pgsql/src/include/postgres.h:49 [m [K,
> >>                  from  [01m [K../pgsql/src/port/pg_numa.c:16 [m [K:
> >>  [01m [K../pgsql/src/include/utils/elog.h:79:10: [m [K
> >>  [01;31m [Kfatal error:  [m [Kutils/errcodes.h: No such file or
> >> directory
> >>    79 | #include  [01;31m [K"utils/errcodes.h" [m [K
> >>       |           [01;31m [K^~~~~~~~~~~~~~~~~~ [m [K
> >> compilation terminated.
> > 
> > $ ninja -t missingdeps
> > Missing dep: src/port/libpgport.a.p/pg_numa.c.o uses src/include/utils/errcodes.h (generated by CUSTOM_COMMAND)
> > Missing dep: src/port/libpgport_shlib.a.p/pg_numa.c.o uses src/include/utils/errcodes.h (generated by CUSTOM_COMMAND)
> > Processed 2384 nodes.
> > Error: There are 2 missing dependency paths.
> > 2 targets had depfile dependencies on 1 distinct generated inputs (from 1 rules)  without a non-depfile dep path to the generator.
> > There might be build flakiness if any of the targets listed above are built alone, or not late enough, in a clean output directory.
> > 
> 
> Wouldn't it be good to add this (ninja -t missingdeps) to the CI task? I
> ran those tests many times, and had it failed at least once I'd have
> fixed it before commit.

Yes, we should. It's a somewhat newer feature, so we originally couldn't.
There only was a very clunky and slow python script when I was doing most of
the meson work.

I was actually thinking that it might make sense as a meson-registered test,
that way one quickly can find the issue both locally and on CI.

Greetings,

Andres Freund



Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. doc: Correct pg_shmem_allocations_numa.size data type

  2. Add pg_buffercache_numa view with NUMA node info

  3. Add support for basic NUMA awareness

  4. Introduce pg_shmem_allocations_numa view