Re: About to add WAL write/fsync statistics to pg_stat_wal view

Japin Li <japinli@hotmail.com>

From: Li Japin <japinli@hotmail.com>
To: Masahiro Ikeda <ikedamsh@oss.nttdata.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2020-12-08T07:45:52Z
Lists: pgsql-hackers
Hi,

> On Dec 8, 2020, at 1:06 PM, Masahiro Ikeda <ikedamsh@oss.nttdata.com> wrote:
> 
> Hi,
> 
> I propose to add wal write/fsync statistics to pg_stat_wal view.
> It's useful not only for developing/improving source code related to WAL
> but also for users to detect workload changes, HW failure, and so on.
> 
> I introduce "track_wal_io_timing" parameter and provide the following information on pg_stat_wal view.
> I separate the parameter from "track_io_timing" to "track_wal_io_timing"
> because IIUC, WAL I/O activity may have a greater impact on query performance than database I/O activity.
> 
> ```
> postgres=# SELECT wal_write, wal_write_time, wal_sync, wal_sync_time FROM pg_stat_wal;
> -[ RECORD 1 ]--+----
> wal_write      | 650  # Total number of times WAL data was written to the disk
> 
> wal_write_time | 43   # Total amount of time that has been spent in the portion of WAL data was written to disk
>                      # if track-wal-io-timing is enabled, otherwise zero
> 
> wal_sync       | 78   # Total number of times WAL data was synced to the disk
> 
> wal_sync_time  | 104  # Total amount of time that has been spent in the portion of WAL data was synced to disk
>                      # if track-wal-io-timing is enabled, otherwise zero
> ```
> 
> What do you think?
> Please let me know your comments.
> 
> Regards
> -- 
> Masahiro Ikeda
> NTT DATA CORPORATION<0001_add_wal_io_activity_to_the_pg_stat_wal.patch>

There is a no previous prototype warning for ‘fsyncMethodCalled’, and it now only used in xlog.c,
should we declare with static? And this function wants a boolean as a return, should we use
true/false other than 0/1?

+/*
+ * Check if fsync mothod is called.
+ */
+bool
+fsyncMethodCalled()
+{
+       if (!enableFsync)
+               return 0;
+
+       switch (sync_method)
+       {
+               case SYNC_METHOD_FSYNC:
+               case SYNC_METHOD_FSYNC_WRITETHROUGH:
+               case SYNC_METHOD_FDATASYNC:
+                       return 1;
+               default:
+                       /* others don't have a specific fsync method */
+                       return 0;
+       }
+}
+

--
Best regards
ChengDu WenWu Information Technology Co.,Ltd.
Japin Li

Commits

  1. Send statistics collected during shutdown checkpoint to the stats collector.

  2. Force to send remaining WAL stats to the stats collector at walwriter exit.

  3. Track total amounts of times spent writing and syncing WAL data to disk.

  4. Retry short writes when flushing WAL.