Re: per backend I/O statistics
Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
To: Alexander Lakhin <exclusion@gmail.com>
Cc: Michael Paquier <michael@paquier.xyz>, Nazir Bilal Yavuz <byavuz81@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, pgsql-hackers@lists.postgresql.org
Date: 2024-12-20T09:09:00Z
Lists: pgsql-hackers
Attachments
- v1-0001-Fix-per-backend-IO-stats-regression-test.patch (text/x-diff)
Hi,
On Fri, Dec 20, 2024 at 08:00:00AM +0200, Alexander Lakhin wrote:
> Hello Michael,
>
> 19.12.2024 06:21, Michael Paquier wrote:
> > Fixed that, bumped the two version counters, and done.
>
> Could you, please, look at recent failures produced by grassquit (which
> has fsync = on in it's config), on an added test case? For instance, [1]:
> --- /home/bf/bf-build/grassquit/HEAD/pgsql/src/test/regress/expected/stats.out 2024-12-19 04:44:08.779311933 +0000
> +++ /home/bf/bf-build/grassquit/HEAD/pgsql.build/testrun/recovery/027_stream_regress/data/results/stats.out
> 2024-12-19 16:37:41.351784840 +0000
> @@ -1333,7 +1333,7 @@
> AND :my_io_sum_shared_after_fsyncs= 0);
> ?column?
> ----------
> - t
> + f
> (1 row)
>
> The complete query is:
> SELECT current_setting('fsync') = 'off'
> OR (:my_io_sum_shared_after_fsyncs = :my_io_sum_shared_before_fsyncs
> AND :my_io_sum_shared_after_fsyncs= 0);
>
> And the corresponding query in 027_stream_regress_primary.log is:
> 2024-12-19 16:37:39.907 UTC [4027467][client backend][15/1980:0] LOG: statement: SELECT current_setting('fsync') = 'off'
> OR (1 = 1
> AND 1= 0);
>
> (I can reproduce this locally with an asan-enabled build too.)
>
> [1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=grassquit&dt=2024-12-19%2016%3A28%3A58
Thanks for the report! I was not able able to reproduce (even with asan-enabled)
but I think the test is wrong. Indeed the backend could fsync too during the test
(see register_dirty_segment() and the case where the request queue is full).
I think the test should look like the attached instead, thoughts?
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Commits
-
Rework handling of pending data for backend statistics
- 4feba03d8b92 18.0 landed
-
Rename some pgstats callbacks related to flush of entries
- 28de66cee5f4 18.0 landed
-
Relax regression test for fsync check of backend-level stats
- 546371599e76 18.0 landed
-
Add backend-level statistics to pgstats
- 9aea73fc61d4 18.0 landed
-
Extract logic filling pg_stat_get_io()'s tuplestore into its own routine
- ff7c40d7fd6a 18.0 landed
-
Tweak some comments related to variable-numbered stats in pgstat.c
- fee2b3ea2ecd 18.0 landed