Re: per backend I/O statistics

Bertrand Drouvot <bertranddrouvot.pg@gmail.com>

From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
To: Alexander Lakhin <exclusion@gmail.com>
Cc: Michael Paquier <michael@paquier.xyz>, Nazir Bilal Yavuz <byavuz81@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, pgsql-hackers@lists.postgresql.org
Date: 2024-12-20T09:09:00Z
Lists: pgsql-hackers

Attachments

Hi,

On Fri, Dec 20, 2024 at 08:00:00AM +0200, Alexander Lakhin wrote:
> Hello Michael,
> 
> 19.12.2024 06:21, Michael Paquier wrote:
> > Fixed that, bumped the two version counters, and done.
> 
> Could you, please, look at recent failures produced by grassquit (which
> has fsync = on in it's config), on an added test case? For instance, [1]:
> --- /home/bf/bf-build/grassquit/HEAD/pgsql/src/test/regress/expected/stats.out 2024-12-19 04:44:08.779311933 +0000
> +++ /home/bf/bf-build/grassquit/HEAD/pgsql.build/testrun/recovery/027_stream_regress/data/results/stats.out
> 2024-12-19 16:37:41.351784840 +0000
> @@ -1333,7 +1333,7 @@
>        AND :my_io_sum_shared_after_fsyncs= 0);
>   ?column?
>  ----------
> - t
> + f
>  (1 row)
> 
> The complete query is:
> SELECT current_setting('fsync') = 'off'
>   OR (:my_io_sum_shared_after_fsyncs = :my_io_sum_shared_before_fsyncs
>       AND :my_io_sum_shared_after_fsyncs= 0);
> 
> And the corresponding query in 027_stream_regress_primary.log is:
> 2024-12-19 16:37:39.907 UTC [4027467][client backend][15/1980:0] LOG:  statement: SELECT current_setting('fsync') = 'off'
>       OR (1 = 1
>           AND 1= 0);
> 
> (I can reproduce this locally with an asan-enabled build too.)
> 
> [1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=grassquit&dt=2024-12-19%2016%3A28%3A58

Thanks for the report! I was not able able to reproduce (even with asan-enabled)
but I think the test is wrong. Indeed the backend could fsync too during the test
(see register_dirty_segment() and the case where the request queue is full).

I think the test should look like the attached instead, thoughts?

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Commits

  1. Rework handling of pending data for backend statistics

  2. Rename some pgstats callbacks related to flush of entries

  3. Relax regression test for fsync check of backend-level stats

  4. Add backend-level statistics to pgstats

  5. Extract logic filling pg_stat_get_io()'s tuplestore into its own routine

  6. Tweak some comments related to variable-numbered stats in pgstat.c