Horton
Menu

Re: Reduce timing overhead of EXPLAIN ANALYZE using rdtsc?

Andres Freund <andres@anarazel.de>

View thread
From: Andres Freund <andres@anarazel.de>
To: Lukas Fittl <lukas@fittl.com>
Cc: John Naylor <johncnaylorls@gmail.com>, Jakub Wartak <jakub.wartak@enterprisedb.com>, Hannu Krosing <hannuk@google.com>, Robert Haas <robertmhaas@gmail.com>, Pavel Stehule <pavel.stehule@gmail.com>, vignesh C <vignesh21@gmail.com>, Michael Paquier <michael@paquier.xyz>, Ibrar Ahmed <ibrar.ahmad@gmail.com>, Maciek Sakrejda <m.sakrejda@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, David Geier <geidav.pg@gmail.com>
Date: 2026-03-08T16:39:47Z
Lists: pgsql-hackers 
Hi,

On 2026-03-06 11:47:10 -0800, Lukas Fittl wrote:
> > But maybe we should just do the stupid thing and figure out the multiplier as
> > such:
> >
> >   ns_to_cycles = tsc_via_rdtsc / to_ns(clock_gettime(CLOCK_BOOTTIME))
> >
> > in some quick experiments that ends up with a very good estimate.  There would
> > have to be an awful long gap between the rdtsc and clock_gettime() computation
> > for the frequency to be meaningfully inaccurate.
> 
> I think as long as the TSC counter and the clock boottime start at the
> same moment, that should work. But I'm not sure if we can rely on that
> to be the case in virtualized environments? I can do some more
> testing.

I did some testing, and unfortunately it's not good enough. There are several
issues:

- The tsc counter starts earlier than the OS, by enough to make counter
  initially not quite right. It's not that bad on a laptop with a quick boot
  time, but on a server with slower bios time initialization (e.g. due to
  training of more memory) it's worse.

- If the server is rebooted not through a hard reset (the typical default),
  but through something like kexec (which does not go through bios again), the
  tsc counter is not reset.



> Alternatively, we could consider doing it like the Kernel does it for
> its calibration loop, and wait 1 second of wall time, and then see how
> far the TSC counter has advanced.

Yea, I think we need a calibration loop, unfortunately. But I think it should
be doable to make it a lot quicker than waiting one second.  I'm thinking of
something like a loop that measures the the clock cycles and relative time
(using clock_gettime()) since the start and does so until the frequency
estimate predicts the time results closely.  I think should be a few 10s of
milliseconds at most.


> FWIW, I ended up getting an x86 machine to be able to test these
> things better, and got myself an AMD CPU.

Dedication...


> Well, turns out that my
> non-virtualized AMD CPU ("AMD Ryzen™ AI Max+ 395") does not provide
> the TSC frequency via CPUID, at all :(

I can repro that on a somewhat older Zen 4 (7840U) laptop CPU.


> Instead on newer AMD CPUs you can use an MSR to get the TSC frequency,
> see [2]

:(

Greetings,

Andres Freund

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. pg_test_timing: Show additional TSC clock source debug info

  2. instrumentation: Avoid CPUID 0x15/0x16 for Hypervisor TSC frequency

  3. pg_test_timing: Also test RDTSC[P] timing, report time source, TSC frequency

  4. Allow retrieving x86 TSC frequency/flags from CPUID

  5. instrumentation: Standardize ticks to nanosecond conversion method

  6. instrumentation: Use Time-Stamp Counter on x86-64 to lower overhead

  7. Check for __cpuidex and __get_cpuid_count separately

  8. pg_test_timing: Reduce per-loop overhead

  9. Refactor handling of x86 CPUID instructions

  10. instrumentation: Drop INSTR_TIME_SET_CURRENT_LAZY macro

  11. Rename pg_crc32c_sse42_choose.c for general purpose

  12. Zero initialize uses of instr_time about to trigger compiler warnings

  13. instr_time: Represent time as an int64 on all platforms

  14. Add 250c8ee07ed to git-blame-ignore-revs