v33-0003-Add-system-view-tracking-IO-ops-per-backend-type.patch
text/x-patch
Filename: v33-0003-Add-system-view-tracking-IO-ops-per-backend-type.patch
Type: text/x-patch
Part: 2
Patch
Format: format-patch
Series: patch v33-0003
Subject: Add system view tracking IO ops per backend type
| File | + | − |
|---|---|---|
| doc/src/sgml/monitoring.sgml | 213 | 2 |
| src/backend/catalog/system_views.sql | 15 | 0 |
| src/backend/utils/adt/pgstatfuncs.c | 130 | 0 |
| src/include/catalog/pg_proc.dat | 9 | 0 |
| src/test/regress/expected/rules.out | 12 | 0 |
| src/test/regress/expected/stats.out | 264 | 0 |
| src/test/regress/sql/stats.sql | 137 | 0 |
From 2f63439e189eff432bba4abfc12762e7c3acd8ea Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 6 Oct 2022 12:24:42 -0400
Subject: [PATCH v33 3/4] Add system view tracking IO ops per backend type
Add pg_stat_io, a system view which tracks the number of IOOps
(clocksweeps, reuses, hits, reads, writes, extends, and fsyncs) done
through each IOContext (shared buffers, local buffers, and buffers
reserved by a BufferAccessStrategy) by each type of backend (e.g. client
backend, checkpointer).
Some BackendTypes do not accumulate IO operations statistics and will
not be included in the view.
Some IOContexts are not used by some BackendTypes and will not be in the
view. For example, checkpointer does not use a BufferAccessStrategy
(currently), so there will be no rows for BufferAccessStrategy
IOContexts for checkpointer.
Some IOOps are invalid in combination with certain IOContexts. Those
cells will be NULL in the view to distinguish between 0 observed IOOps
of that type and an invalid combination. For example, local buffers are
not fsynced so cells for all BackendTypes for IOCONTEXT_LOCAL and
IOOP_FSYNC will be NULL.
Some BackendTypes never perform certain IOOps. Those cells will also be
NULL in the view. For example, bgwriter should not perform reads.
View stats are populated with statistics incremented when a backend
performs an IO Operation and maintained by the cumulative statistics
subsystem.
Each row of the view shows stats for a particular BackendType and
IOContext combination (e.g. shared buffer accesses by checkpointer) and
each column in the view is the total number of IO Operations done (e.g.
writes).
So a cell in the view would be, for example, the number of shared
buffers written by checkpointer since the last stats reset.
In anticipation of tracking WAL IO and non-block-oriented IO (such as
temporary file IO), the "unit" column specifies the unit of the "read",
"written", and "extended" columns for a given row.
Note that some of the cells in the view are redundant with fields in
pg_stat_bgwriter (e.g. buffers_backend), however these have been kept in
pg_stat_bgwriter for backwards compatibility. Deriving the redundant
pg_stat_bgwriter stats from the IO operations stats structures was also
problematic due to the separate reset targets for 'bgwriter' and 'io'.
Suggested by Andres Freund
Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Maciek Sakrejda <m.sakrejda@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Discussion: https://www.postgresql.org/message-id/flat/20200124195226.lth52iydq2n2uilq%40alap3.anarazel.de
---
doc/src/sgml/monitoring.sgml | 215 +++++++++++++++++++++-
src/backend/catalog/system_views.sql | 15 ++
src/backend/utils/adt/pgstatfuncs.c | 130 +++++++++++++
src/include/catalog/pg_proc.dat | 9 +
src/test/regress/expected/rules.out | 12 ++
src/test/regress/expected/stats.out | 264 +++++++++++++++++++++++++++
src/test/regress/sql/stats.sql | 137 ++++++++++++++
7 files changed, 780 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 14dfd650f8..926eb40f75 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -448,6 +448,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</entry>
</row>
+ <row>
+ <entry><structname>pg_stat_io</structname><indexterm><primary>pg_stat_io</primary></indexterm></entry>
+ <entry>A row for each IO Context for each backend type showing
+ statistics about backend IO operations. See
+ <link linkend="monitoring-pg-stat-io-view">
+ <structname>pg_stat_io</structname></link> for details.
+ </entry>
+ </row>
+
<row>
<entry><structname>pg_stat_wal</structname><indexterm><primary>pg_stat_wal</primary></indexterm></entry>
<entry>One row only, showing statistics about WAL activity. See
@@ -3600,13 +3609,12 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
<structfield>stats_reset</structfield> <type>timestamp with time zone</type>
</para>
<para>
- Time at which these statistics were last reset
+ Time at which these statistics were last reset.
</para></entry>
</row>
</tbody>
</tgroup>
</table>
-
<para>
Normally, WAL files are archived in order, oldest to newest, but that is
not guaranteed, and does not hold under special circumstances like when
@@ -3615,7 +3623,210 @@ SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event i
<structfield>last_archived_wal</structfield> have also been successfully
archived.
</para>
+ </sect2>
+
+ <sect2 id="monitoring-pg-stat-io-view">
+ <title><structname>pg_stat_io</structname></title>
+
+ <indexterm>
+ <primary>pg_stat_io</primary>
+ </indexterm>
+
+ <para>
+ The <structname>pg_stat_io</structname> view has a row for each backend type
+ and IO context containing global data for the cluster on IO operations done
+ by that backend type in that IO context. Currently, only a subset of IO
+ operations are tracked here. WAL IO, IO on temporary files, and some forms
+ of IO outside of shared buffers (such as when building indexes or moving a
+ table from one tablespace to another) could be added in the future.
+ </para>
+
+ <table id="pg-stat-io-view" xreflabel="pg_stat_io">
+ <title><structname>pg_stat_io</structname> View</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>backend_type</structfield> <type>text</type>
+ </para>
+ <para>
+ Type of backend (e.g. background worker, autovacuum worker).
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>io_context</structfield> <type>text</type>
+ </para>
+ <para>
+ IO Context used. This refers to the context or location of an IO
+ operation.
+ <literal>shared</literal> refers to shared buffers, the primary
+ buffer pool for relation data.
+ <literal>local</literal> refers to
+ process-local memory used for temporary tables.
+ <literal>vacuum</literal> refers to memory reserved for use during
+ vacuumming and analyzing.
+ <literal>bulkread</literal>
+ refers to memory reserved for use during bulk read operations.
+ <literal>bulkwrite</literal>
+ refers to memory reserved for use during bulk write operations.
+ The autovacuum daemon, explicit <command>VACUUM</command>, explicit
+ <command>ANALYZE</command>, many bulk reads, and many bulk writes use a
+ fixed amount of memory, acquiring the equivalent number of shared
+ buffers and reusing them circularly to avoid occupying an undue portion
+ of the main shared buffer pool.
+ </para></entry>
+ </row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>read</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Reads by this <varname>backend_type</varname> into
+ memory or buffers in this <varname>io_context</varname>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>written</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Writes of data in this <varname>io_context</varname> written out by this
+ <varname>backend_type</varname>.
+ Note that the values of <varname>written</varname> for
+ <varname>backend_type</varname> <literal>background writer</literal> and
+ <varname>backend_type</varname> <literal>checkpointer</literal> are
+ equivalent to the values of <varname>buffers_clean</varname> and
+ <varname>buffers_checkpoint</varname>, respectively, in <link
+ linkend="monitoring-pg-stat-bgwriter-view">
+ <structname>pg_stat_bgwriter</structname></link>.
+ Also, the sum of <varname>written</varname> and
+ <varname>extended</varname> in this view for
+ <varname>backend_type</varname>s <literal>client backend</literal>,
+ <literal>autovacuum worker</literal>, <literal>background
+ worker</literal>, and <literal>walsender</literal> in
+ <varname>io_context</varname>s <literal>shared</literal>,
+ <literal>bulkread</literal>, <literal>bulkwrite</literal>, and
+ <literal>vacuum</literal> is equivalent to
+ <varname>buffers_backend</varname> in
+ <structname>pg_stat_bgwriter</structname>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>extended</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Extends of relations done by this <varname>backend_type</varname> in
+ order to write data in this <varname>io_context</varname>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>unit</structfield> <type>text</type>
+ </para>
+ <para>
+ The unit of IO read, written, or extended. Currently
+ <varname>block_size</varname> is the only possible value. Reads, writes,
+ and extends of relation data are done in <varname>block_size</varname>
+ units. Future values could include <varname>wal_block_size</varname>,
+ once WAL IO is tracked in this view, and <quote>bytes</quote>, once
+ non-block-oriented IO such as temporary file IO is tracked here.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>clocksweeps</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of clocksweeps done by this <varname>backend_type</varname> in
+ order to acquire a buffer in this <varname>io_context</varname>. A
+ <literal>clocksweep</literal> in the <literal>bulkread</literal>,
+ <literal>bulkwrite</literal>, or <literal>vacuum</literal>
+ <varname>io_context</varname>s is counted when a backend adds a shared
+ buffer to the fixed-size ring used to avoid consuming excessive shared
+ buffers. If the backend has pinned all of the buffers in the ring, it
+ may add a replacement shared buffer to the ring. This will also be
+ counted as a <literal>clocksweep</literal>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>reused</structfield> <type>bigint</type>
+ </para>
+ <para>
+ The number of times a buffer was reused as part of an operation in the
+ <literal>bulkread</literal>, <literal>bulkwrite</literal>, and
+ <literal>vacuum</literal> <varname>io_context</varname>s.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>hit</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Relevant only for block-based IO of data accessed in the course of
+ satisfying queries, <varname>hit</varname> is the number of number of
+ accesses of blocks already located in a
+ <productname>PostgreSQL</productname> buffer in this specified
+ <varname>io_context</varname> by this <varname>backend_type</varname>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>files_synced</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of files fsynced by this <varname>backend_type</varname> for the
+ purpose of persisting data dirtied in this
+ <varname>io_context</varname>. <literal>fsyncs</literal> are done at
+ segment boundaries so <varname>unit</varname> does not apply to the
+ <varname>files_synced</varname> column. <literal>fsyncs</literal> done
+ by backends in order to persist data written in
+ <varname>io_context</varname> <literal>vacuum</literal>,
+ <varname>io_context</varname> <literal>bulkread</literal>, or
+ <varname>io_context</varname> <literal>bulkwrite</literal> are counted
+ as an <varname>io_context</varname> <literal>shared</literal>
+ <literal>fsync</literal>. Note that the sum of
+ <varname>files_synced</varname> for all <varname>io_context</varname>
+ <literal>shared</literal> for all <varname>backend_type</varname>s
+ except <literal>checkpointer</literal> is equivalent to
+ <varname>buffers_backend_fsync</varname> in <link
+ linkend="monitoring-pg-stat-bgwriter-view">
+ <structname>pg_stat_bgwriter</structname></link>.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>stats_reset</structfield> <type>timestamp with time zone</type>
+ </para>
+ <para>
+ Time at which these statistics were last reset.
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
</sect2>
<sect2 id="monitoring-pg-stat-bgwriter-view">
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 55f7ec79e0..25e0cef114 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1114,6 +1114,21 @@ CREATE VIEW pg_stat_bgwriter AS
pg_stat_get_buf_alloc() AS buffers_alloc,
pg_stat_get_bgwriter_stat_reset_time() AS stats_reset;
+CREATE VIEW pg_stat_io AS
+SELECT
+ b.backend_type,
+ b.io_context,
+ b.read,
+ b.written,
+ b.extended,
+ b.unit,
+ b.clocksweeps,
+ b.reused,
+ b.hit,
+ b.files_synced,
+ b.stats_reset
+FROM pg_stat_get_io() b;
+
CREATE VIEW pg_stat_wal AS
SELECT
w.wal_records,
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index edd73e5c25..62fbf7e53a 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1712,6 +1712,136 @@ pg_stat_get_buf_alloc(PG_FUNCTION_ARGS)
PG_RETURN_INT64(pgstat_fetch_stat_bgwriter()->buf_alloc);
}
+/*
+* When adding a new column to the pg_stat_io view, add a new enum value
+* here above IO_NUM_COLUMNS.
+*/
+typedef enum io_stat_col
+{
+ IO_COL_BACKEND_TYPE,
+ IO_COL_IO_CONTEXT,
+ IO_COL_READS,
+ IO_COL_WRITES,
+ IO_COL_EXTENDS,
+ IO_COL_UNIT,
+ IO_COL_CLOCKSWEEPS,
+ IO_COL_REUSES,
+ IO_COL_HITS,
+ IO_COL_FSYNCS,
+ IO_COL_RESET_TIME,
+ IO_NUM_COLUMNS,
+} io_stat_col;
+
+/*
+ * When adding a new IOOp, add a new io_stat_col and add a case to this
+ * function returning the corresponding io_stat_col.
+ */
+static io_stat_col
+pgstat_io_op_get_index(IOOp io_op)
+{
+ switch (io_op)
+ {
+ case IOOP_CLOCKSWEEP:
+ return IO_COL_CLOCKSWEEPS;
+ case IOOP_HIT:
+ return IO_COL_HITS;
+ case IOOP_READ:
+ return IO_COL_READS;
+ case IOOP_REUSE:
+ return IO_COL_REUSES;
+ case IOOP_WRITE:
+ return IO_COL_WRITES;
+ case IOOP_EXTEND:
+ return IO_COL_EXTENDS;
+ case IOOP_FSYNC:
+ return IO_COL_FSYNCS;
+ }
+
+ elog(ERROR, "unrecognized IOOp value: %d", io_op);
+}
+
+Datum
+pg_stat_get_io(PG_FUNCTION_ARGS)
+{
+ PgStat_BackendIOContextOps *backends_io_stats;
+ ReturnSetInfo *rsinfo;
+ Datum reset_time;
+
+ SetSingleFuncCall(fcinfo, 0);
+ rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+
+ backends_io_stats = pgstat_fetch_backend_io_context_ops();
+
+ reset_time = TimestampTzGetDatum(backends_io_stats->stat_reset_timestamp);
+
+ for (int bktype = 0; bktype < BACKEND_NUM_TYPES; bktype++)
+ {
+ Datum bktype_desc = CStringGetTextDatum(GetBackendTypeDesc((BackendType) bktype));
+ bool expect_backend_stats = true;
+ PgStat_IOContextOps *io_context_ops = &backends_io_stats->stats[bktype];
+
+ /*
+ * For those BackendTypes without IO Operation stats, skip
+ * representing them in the view altogether.
+ */
+ expect_backend_stats = pgstat_io_op_stats_collected((BackendType)
+ bktype);
+
+ for (int io_context = 0; io_context < IOCONTEXT_NUM_TYPES; io_context++)
+ {
+ PgStat_IOOpCounters *counters = &io_context_ops->data[io_context];
+ const char *io_context_str = pgstat_io_context_desc(io_context);
+
+ Datum values[IO_NUM_COLUMNS] = {0};
+ bool nulls[IO_NUM_COLUMNS] = {0};
+
+ /*
+ * Some combinations of IOContext and BackendType are not valid
+ * for any type of IOOp. In such cases, omit the entire row from
+ * the view.
+ */
+ if (!expect_backend_stats ||
+ !pgstat_bktype_io_context_valid((BackendType) bktype,
+ (IOContext) io_context))
+ {
+ pgstat_io_context_ops_assert_zero(counters);
+ continue;
+ }
+
+ values[IO_COL_BACKEND_TYPE] = bktype_desc;
+ values[IO_COL_IO_CONTEXT] = CStringGetTextDatum(io_context_str);
+ values[IO_COL_READS] = Int64GetDatum(counters->reads);
+ values[IO_COL_WRITES] = Int64GetDatum(counters->writes);
+ values[IO_COL_EXTENDS] = Int64GetDatum(counters->extends);
+ values[IO_COL_UNIT] = CStringGetTextDatum("block_size");
+ values[IO_COL_CLOCKSWEEPS] = Int64GetDatum(counters->clocksweeps);
+ values[IO_COL_REUSES] = Int64GetDatum(counters->reuses);
+ values[IO_COL_HITS] = Int64GetDatum(counters->hits);
+ values[IO_COL_FSYNCS] = Int64GetDatum(counters->fsyncs);
+ values[IO_COL_RESET_TIME] = TimestampTzGetDatum(reset_time);
+
+ /*
+ * Some combinations of BackendType and IOOp and of IOContext and
+ * IOOp are not valid. Set these cells in the view NULL and assert
+ * that these stats are zero as expected.
+ */
+ for (int io_op = 0; io_op < IOOP_NUM_TYPES; io_op++)
+ {
+ if (!(pgstat_io_op_valid((BackendType) bktype, (IOContext)
+ io_context, (IOOp) io_op)))
+ {
+ pgstat_io_op_assert_zero(counters, (IOOp) io_op);
+ nulls[pgstat_io_op_get_index((IOOp) io_op)] = true;
+ }
+ }
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+ }
+ }
+
+ return (Datum) 0;
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 68bb032d3e..77edbe9517 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5649,6 +5649,15 @@
proname => 'pg_stat_get_buf_alloc', provolatile => 's', proparallel => 'r',
prorettype => 'int8', proargtypes => '', prosrc => 'pg_stat_get_buf_alloc' },
+{ oid => '8459', descr => 'statistics: per backend type IO statistics',
+ proname => 'pg_stat_get_io', provolatile => 'v',
+ prorows => '14', proretset => 't',
+ proparallel => 'r', prorettype => 'record', proargtypes => '',
+ proallargtypes => '{text,text,int8,int8,int8,text,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_type,io_context,read,written,extended,unit,clocksweeps,reused,hit,files_synced,stats_reset}',
+ prosrc => 'pg_stat_get_io' },
+
{ oid => '1136', descr => 'statistics: information about WAL activity',
proname => 'pg_stat_get_wal', proisstrict => 'f', provolatile => 's',
proparallel => 'r', prorettype => 'record', proargtypes => '',
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 9dd137415e..7cedd530f5 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1868,6 +1868,18 @@ pg_stat_gssapi| SELECT s.pid,
s.gss_enc AS encrypted
FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, leader_pid, query_id)
WHERE (s.client_port IS NOT NULL);
+pg_stat_io| SELECT b.backend_type,
+ b.io_context,
+ b.read,
+ b.written,
+ b.extended,
+ b.unit,
+ b.clocksweeps,
+ b.reused,
+ b.hit,
+ b.files_synced,
+ b.stats_reset
+ FROM pg_stat_get_io() b(backend_type, io_context, read, written, extended, unit, clocksweeps, reused, hit, files_synced, stats_reset);
pg_stat_progress_analyze| SELECT s.pid,
s.datid,
d.datname,
diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out
index f701da2069..0172a6c95e 100644
--- a/src/test/regress/expected/stats.out
+++ b/src/test/regress/expected/stats.out
@@ -918,4 +918,268 @@ SELECT pg_stat_get_subscription_stats(NULL);
(1 row)
+-- Test that the following operations are tracked in pg_stat_io:
+-- - clocksweeps of shared buffers
+-- - reads of target blocks into shared buffers
+-- - shared buffer cache hits when target blocks reside in shared buffers
+-- - writes of shared buffers
+-- - extends of relations using shared buffers
+-- - fsyncs done to ensure the durability of data dirtying shared buffers
+SELECT sum(clocksweeps) AS io_sum_shared_clocksweeps_before FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(hit) AS io_sum_shared_hits_before FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(read) AS io_sum_shared_reads_before FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(written) AS io_sum_shared_writes_before FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(extended) AS io_sum_shared_extends_before FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(files_synced) AS io_sum_shared_fsyncs_before FROM pg_stat_io WHERE io_context = 'shared' \gset
+-- Create a regular table and insert some data to generate IOCONTEXT_SHARED
+-- clocksweeps and extends.
+CREATE TABLE test_io_shared(a int);
+INSERT INTO test_io_shared SELECT i FROM generate_series(1,100)i;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+-- After a checkpoint, there should be some additional IOCONTEXT_SHARED writes
+-- and fsyncs.
+-- The second checkpoint ensures that stats from the first checkpoint have been
+-- reported and protects against any potential races amongst the table
+-- creation, a possible timing-triggered checkpoint, and the explicit
+-- checkpoint in the test.
+CHECKPOINT;
+CHECKPOINT;
+SELECT sum(clocksweeps) AS io_sum_shared_clocksweeps_after FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(written) AS io_sum_shared_writes_after FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(extended) AS io_sum_shared_extends_after FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(files_synced) AS io_sum_shared_fsyncs_after FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT :io_sum_shared_clocksweeps_after > :io_sum_shared_clocksweeps_before;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT :io_sum_shared_writes_after > :io_sum_shared_writes_before;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT :io_sum_shared_extends_after > :io_sum_shared_extends_before;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT current_setting('fsync') = 'off' OR :io_sum_shared_fsyncs_after > :io_sum_shared_fsyncs_before;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Change the tablespace so that the table is rewritten directly, then SELECT
+-- from it to cause it to be read back into shared buffers.
+SET allow_in_place_tablespaces = true;
+CREATE TABLESPACE test_io_shared_stats_tblspc LOCATION '';
+ALTER TABLE test_io_shared SET TABLESPACE test_io_shared_stats_tblspc;
+SELECT COUNT(*) FROM test_io_shared;
+ count
+-------
+ 100
+(1 row)
+
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT sum(read) AS io_sum_shared_reads_after FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT :io_sum_shared_reads_after > :io_sum_shared_reads_before;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Select from the table again once it is in shared buffers. There should be
+-- some hits recorded in pg_stat_io.
+SELECT sum(hit) AS io_sum_shared_hits_after FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT :io_sum_shared_hits_after > :io_sum_shared_hits_before;
+ ?column?
+----------
+ t
+(1 row)
+
+DROP TABLE test_io_shared;
+DROP TABLESPACE test_io_shared_stats_tblspc;
+-- Test that clocksweeps of local buffers, reads of temporary table blocks
+-- into local buffers, temporary table block cache hits in local buffers,
+-- writes of local buffers, and extends of temporary tables are tracked in
+-- pg_stat_io.
+-- Set temp_buffers to a low value so that we can trigger writes with fewer
+-- inserted tuples. Do so in a new session in case temporary tables have been
+-- accessed by previous tests in this session.
+\c
+SET temp_buffers TO '1MB';
+CREATE TEMPORARY TABLE test_io_local(a int, b TEXT);
+SELECT sum(clocksweeps) AS io_sum_local_clocksweeps_before FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(hit) AS io_sum_local_hits_before FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(read) AS io_sum_local_reads_before FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(written) AS io_sum_local_writes_before FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(extended) AS io_sum_local_extends_before FROM pg_stat_io WHERE io_context = 'local' \gset
+-- Insert enough values that we need to reuse and write out dirty local
+-- buffers.
+INSERT INTO test_io_local SELECT generate_series(1, 8000) as id, repeat('a', 100);
+-- Read in evicted buffers.
+SELECT COUNT(*) FROM test_io_local;
+ count
+-------
+ 8000
+(1 row)
+
+-- Query tuples in local buffers to ensure new local buffer cache hits.
+SELECT COUNT(*) FROM test_io_local;
+ count
+-------
+ 8000
+(1 row)
+
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT sum(clocksweeps) AS io_sum_local_clocksweeps_after FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(hit) AS io_sum_local_hits_after FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(read) AS io_sum_local_reads_after FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(written) AS io_sum_local_writes_after FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(extended) AS io_sum_local_extends_after FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT :io_sum_local_clocksweeps_after > :io_sum_local_clocksweeps_before;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT :io_sum_local_reads_after > :io_sum_local_reads_before;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT :io_sum_local_hits_after > :io_sum_local_hits_before;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT :io_sum_local_writes_after > :io_sum_local_writes_before;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT :io_sum_local_extends_after > :io_sum_local_extends_before;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET temp_buffers;
+-- Test that reuse of strategy buffers and reads of blocks into these reused
+-- buffers while VACUUMing are tracked in pg_stat_io.
+-- Set wal_skip_threshold smaller than the expected size of
+-- test_io_vac_strategy so that, even if wal_level is minimal, VACUUM FULL will
+-- fsync the newly rewritten test_io_vac_strategy instead of writing it to WAL.
+-- Writing it to WAL will result in the newly written relation pages being in
+-- shared buffers -- preventing us from testing BAS_VACUUM BufferAccessStrategy
+-- clocksweeps and reads.
+SET wal_skip_threshold = '1 kB';
+SELECT sum(clocksweeps) AS io_sum_vac_strategy_clocksweeps_before FROM pg_stat_io WHERE io_context = 'vacuum' \gset
+SELECT sum(reused) AS io_sum_vac_strategy_reuses_before FROM pg_stat_io WHERE io_context = 'vacuum' \gset
+SELECT sum(read) AS io_sum_vac_strategy_reads_before FROM pg_stat_io WHERE io_context = 'vacuum' \gset
+CREATE TABLE test_io_vac_strategy(a int, b int) WITH (autovacuum_enabled = 'false');
+INSERT INTO test_io_vac_strategy SELECT i, i from generate_series(1, 8000)i;
+-- Ensure that the next VACUUM will need to perform IO by rewriting the table
+-- first with VACUUM (FULL).
+VACUUM (FULL) test_io_vac_strategy;
+VACUUM (PARALLEL 0) test_io_vac_strategy;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT sum(clocksweeps) AS io_sum_vac_strategy_clocksweeps_after FROM pg_stat_io WHERE io_context = 'vacuum' \gset
+SELECT sum(reused) AS io_sum_vac_strategy_reuses_after FROM pg_stat_io WHERE io_context = 'vacuum' \gset
+SELECT sum(read) AS io_sum_vac_strategy_reads_after FROM pg_stat_io WHERE io_context = 'vacuum' \gset
+SELECT :io_sum_vac_strategy_clocksweeps_after > :io_sum_vac_strategy_clocksweeps_before;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT :io_sum_vac_strategy_reads_after > :io_sum_vac_strategy_reads_before;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT :io_sum_vac_strategy_reuses_after > :io_sum_vac_strategy_reuses_before;
+ ?column?
+----------
+ t
+(1 row)
+
+RESET wal_skip_threshold;
+-- Test that extends done by a CTAS, which uses a BAS_BULKWRITE
+-- BufferAccessStrategy, are tracked in pg_stat_io.
+SELECT sum(extended) AS io_sum_bulkwrite_strategy_extends_before FROM pg_stat_io WHERE io_context = 'bulkwrite' \gset
+CREATE TABLE test_io_bulkwrite_strategy AS SELECT i FROM generate_series(1,100)i;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT sum(extended) AS io_sum_bulkwrite_strategy_extends_after FROM pg_stat_io WHERE io_context = 'bulkwrite' \gset
+SELECT :io_sum_bulkwrite_strategy_extends_after > :io_sum_bulkwrite_strategy_extends_before;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Test that reads of blocks into reused strategy buffers during database
+-- creation, which uses a BAS_BULKREAD BufferAccessStrategy, are tracked in
+-- pg_stat_io.
+SELECT sum(read) AS io_sum_bulkread_strategy_reads_before FROM pg_stat_io WHERE io_context = 'bulkread' \gset
+CREATE DATABASE test_io_bulkread_strategy_db;
+SELECT pg_stat_force_next_flush();
+ pg_stat_force_next_flush
+--------------------------
+
+(1 row)
+
+SELECT sum(read) AS io_sum_bulkread_strategy_reads_after FROM pg_stat_io WHERE io_context = 'bulkread' \gset
+SELECT :io_sum_bulkread_strategy_reads_after > :io_sum_bulkread_strategy_reads_before;
+ ?column?
+----------
+ t
+(1 row)
+
+-- Test IO stats reset
+SELECT sum(clocksweeps) + sum(reused) + sum(extended) + sum(files_synced) + sum(read) + sum(written) AS io_stats_pre_reset FROM pg_stat_io \gset
+SELECT pg_stat_reset_shared('io');
+ pg_stat_reset_shared
+----------------------
+
+(1 row)
+
+SELECT sum(clocksweeps) + sum(reused) + sum(extended) + sum(files_synced) + sum(read) + sum(written) AS io_stats_post_reset FROM pg_stat_io \gset
+SELECT :io_stats_post_reset < :io_stats_pre_reset;
+ ?column?
+----------
+ t
+(1 row)
+
-- End of Stats Test
diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql
index eb081f65a4..d3860fd9df 100644
--- a/src/test/regress/sql/stats.sql
+++ b/src/test/regress/sql/stats.sql
@@ -449,4 +449,141 @@ SELECT pg_stat_get_replication_slot(NULL);
SELECT pg_stat_get_subscription_stats(NULL);
+-- Test that the following operations are tracked in pg_stat_io:
+-- - clocksweeps of shared buffers
+-- - reads of target blocks into shared buffers
+-- - shared buffer cache hits when target blocks reside in shared buffers
+-- - writes of shared buffers
+-- - extends of relations using shared buffers
+-- - fsyncs done to ensure the durability of data dirtying shared buffers
+SELECT sum(clocksweeps) AS io_sum_shared_clocksweeps_before FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(hit) AS io_sum_shared_hits_before FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(read) AS io_sum_shared_reads_before FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(written) AS io_sum_shared_writes_before FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(extended) AS io_sum_shared_extends_before FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(files_synced) AS io_sum_shared_fsyncs_before FROM pg_stat_io WHERE io_context = 'shared' \gset
+-- Create a regular table and insert some data to generate IOCONTEXT_SHARED
+-- clocksweeps and extends.
+CREATE TABLE test_io_shared(a int);
+INSERT INTO test_io_shared SELECT i FROM generate_series(1,100)i;
+SELECT pg_stat_force_next_flush();
+-- After a checkpoint, there should be some additional IOCONTEXT_SHARED writes
+-- and fsyncs.
+-- The second checkpoint ensures that stats from the first checkpoint have been
+-- reported and protects against any potential races amongst the table
+-- creation, a possible timing-triggered checkpoint, and the explicit
+-- checkpoint in the test.
+CHECKPOINT;
+CHECKPOINT;
+SELECT sum(clocksweeps) AS io_sum_shared_clocksweeps_after FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(written) AS io_sum_shared_writes_after FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(extended) AS io_sum_shared_extends_after FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT sum(files_synced) AS io_sum_shared_fsyncs_after FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT :io_sum_shared_clocksweeps_after > :io_sum_shared_clocksweeps_before;
+SELECT :io_sum_shared_writes_after > :io_sum_shared_writes_before;
+SELECT :io_sum_shared_extends_after > :io_sum_shared_extends_before;
+SELECT current_setting('fsync') = 'off' OR :io_sum_shared_fsyncs_after > :io_sum_shared_fsyncs_before;
+-- Change the tablespace so that the table is rewritten directly, then SELECT
+-- from it to cause it to be read back into shared buffers.
+SET allow_in_place_tablespaces = true;
+CREATE TABLESPACE test_io_shared_stats_tblspc LOCATION '';
+ALTER TABLE test_io_shared SET TABLESPACE test_io_shared_stats_tblspc;
+SELECT COUNT(*) FROM test_io_shared;
+SELECT pg_stat_force_next_flush();
+SELECT sum(read) AS io_sum_shared_reads_after FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT :io_sum_shared_reads_after > :io_sum_shared_reads_before;
+-- Select from the table again once it is in shared buffers. There should be
+-- some hits recorded in pg_stat_io.
+SELECT sum(hit) AS io_sum_shared_hits_after FROM pg_stat_io WHERE io_context = 'shared' \gset
+SELECT :io_sum_shared_hits_after > :io_sum_shared_hits_before;
+DROP TABLE test_io_shared;
+DROP TABLESPACE test_io_shared_stats_tblspc;
+
+-- Test that clocksweeps of local buffers, reads of temporary table blocks
+-- into local buffers, temporary table block cache hits in local buffers,
+-- writes of local buffers, and extends of temporary tables are tracked in
+-- pg_stat_io.
+
+-- Set temp_buffers to a low value so that we can trigger writes with fewer
+-- inserted tuples. Do so in a new session in case temporary tables have been
+-- accessed by previous tests in this session.
+\c
+SET temp_buffers TO '1MB';
+CREATE TEMPORARY TABLE test_io_local(a int, b TEXT);
+SELECT sum(clocksweeps) AS io_sum_local_clocksweeps_before FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(hit) AS io_sum_local_hits_before FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(read) AS io_sum_local_reads_before FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(written) AS io_sum_local_writes_before FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(extended) AS io_sum_local_extends_before FROM pg_stat_io WHERE io_context = 'local' \gset
+-- Insert enough values that we need to reuse and write out dirty local
+-- buffers.
+INSERT INTO test_io_local SELECT generate_series(1, 8000) as id, repeat('a', 100);
+-- Read in evicted buffers.
+SELECT COUNT(*) FROM test_io_local;
+-- Query tuples in local buffers to ensure new local buffer cache hits.
+SELECT COUNT(*) FROM test_io_local;
+SELECT pg_stat_force_next_flush();
+SELECT sum(clocksweeps) AS io_sum_local_clocksweeps_after FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(hit) AS io_sum_local_hits_after FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(read) AS io_sum_local_reads_after FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(written) AS io_sum_local_writes_after FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT sum(extended) AS io_sum_local_extends_after FROM pg_stat_io WHERE io_context = 'local' \gset
+SELECT :io_sum_local_clocksweeps_after > :io_sum_local_clocksweeps_before;
+SELECT :io_sum_local_reads_after > :io_sum_local_reads_before;
+SELECT :io_sum_local_hits_after > :io_sum_local_hits_before;
+SELECT :io_sum_local_writes_after > :io_sum_local_writes_before;
+SELECT :io_sum_local_extends_after > :io_sum_local_extends_before;
+RESET temp_buffers;
+
+-- Test that reuse of strategy buffers and reads of blocks into these reused
+-- buffers while VACUUMing are tracked in pg_stat_io.
+
+-- Set wal_skip_threshold smaller than the expected size of
+-- test_io_vac_strategy so that, even if wal_level is minimal, VACUUM FULL will
+-- fsync the newly rewritten test_io_vac_strategy instead of writing it to WAL.
+-- Writing it to WAL will result in the newly written relation pages being in
+-- shared buffers -- preventing us from testing BAS_VACUUM BufferAccessStrategy
+-- clocksweeps and reads.
+SET wal_skip_threshold = '1 kB';
+SELECT sum(clocksweeps) AS io_sum_vac_strategy_clocksweeps_before FROM pg_stat_io WHERE io_context = 'vacuum' \gset
+SELECT sum(reused) AS io_sum_vac_strategy_reuses_before FROM pg_stat_io WHERE io_context = 'vacuum' \gset
+SELECT sum(read) AS io_sum_vac_strategy_reads_before FROM pg_stat_io WHERE io_context = 'vacuum' \gset
+CREATE TABLE test_io_vac_strategy(a int, b int) WITH (autovacuum_enabled = 'false');
+INSERT INTO test_io_vac_strategy SELECT i, i from generate_series(1, 8000)i;
+-- Ensure that the next VACUUM will need to perform IO by rewriting the table
+-- first with VACUUM (FULL).
+VACUUM (FULL) test_io_vac_strategy;
+VACUUM (PARALLEL 0) test_io_vac_strategy;
+SELECT pg_stat_force_next_flush();
+SELECT sum(clocksweeps) AS io_sum_vac_strategy_clocksweeps_after FROM pg_stat_io WHERE io_context = 'vacuum' \gset
+SELECT sum(reused) AS io_sum_vac_strategy_reuses_after FROM pg_stat_io WHERE io_context = 'vacuum' \gset
+SELECT sum(read) AS io_sum_vac_strategy_reads_after FROM pg_stat_io WHERE io_context = 'vacuum' \gset
+SELECT :io_sum_vac_strategy_clocksweeps_after > :io_sum_vac_strategy_clocksweeps_before;
+SELECT :io_sum_vac_strategy_reads_after > :io_sum_vac_strategy_reads_before;
+SELECT :io_sum_vac_strategy_reuses_after > :io_sum_vac_strategy_reuses_before;
+RESET wal_skip_threshold;
+
+-- Test that extends done by a CTAS, which uses a BAS_BULKWRITE
+-- BufferAccessStrategy, are tracked in pg_stat_io.
+SELECT sum(extended) AS io_sum_bulkwrite_strategy_extends_before FROM pg_stat_io WHERE io_context = 'bulkwrite' \gset
+CREATE TABLE test_io_bulkwrite_strategy AS SELECT i FROM generate_series(1,100)i;
+SELECT pg_stat_force_next_flush();
+SELECT sum(extended) AS io_sum_bulkwrite_strategy_extends_after FROM pg_stat_io WHERE io_context = 'bulkwrite' \gset
+SELECT :io_sum_bulkwrite_strategy_extends_after > :io_sum_bulkwrite_strategy_extends_before;
+
+-- Test that reads of blocks into reused strategy buffers during database
+-- creation, which uses a BAS_BULKREAD BufferAccessStrategy, are tracked in
+-- pg_stat_io.
+SELECT sum(read) AS io_sum_bulkread_strategy_reads_before FROM pg_stat_io WHERE io_context = 'bulkread' \gset
+CREATE DATABASE test_io_bulkread_strategy_db;
+SELECT pg_stat_force_next_flush();
+SELECT sum(read) AS io_sum_bulkread_strategy_reads_after FROM pg_stat_io WHERE io_context = 'bulkread' \gset
+SELECT :io_sum_bulkread_strategy_reads_after > :io_sum_bulkread_strategy_reads_before;
+
+-- Test IO stats reset
+SELECT sum(clocksweeps) + sum(reused) + sum(extended) + sum(files_synced) + sum(read) + sum(written) AS io_stats_pre_reset FROM pg_stat_io \gset
+SELECT pg_stat_reset_shared('io');
+SELECT sum(clocksweeps) + sum(reused) + sum(extended) + sum(files_synced) + sum(read) + sum(written) AS io_stats_post_reset FROM pg_stat_io \gset
+SELECT :io_stats_post_reset < :io_stats_pre_reset;
+
-- End of Stats Test
--
2.34.1