Horton

Re: SLRU statistics

Fujii Masao <masao.fujii@oss.nttdata.com>

View thread
From: Fujii Masao <masao.fujii@oss.nttdata.com>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>, Masahiko Sawada <masahiko.sawada@2ndquadrant.com>, "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2020-05-02T06:56:07Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Fix async.c to not register any SLRU stats counts in the postmaster.

  2. Use proper GetDatum function in pg_stat_get_slru().

  3. Initialize SLRU stats entries to zero.

  4. Remove pg_xact from pg_stat_reset_slru docs

  5. Remove pg_xact entry from SLRU stats

  6. Track SLRU page hits in SimpleLruReadPage_ReadOnly

  7. Fix typo in SLRU stats documentation

  8. Collect statistics about SLRU caches

  9. Simplify and rename some GUC variables, per various recent discussions:


On 2020/05/02 9:08, Tomas Vondra wrote:
> On Fri, May 01, 2020 at 11:49:51AM +0900, Fujii Masao wrote:
>>
>>
>> On 2020/05/01 3:19, Tomas Vondra wrote:
>>> On Fri, May 01, 2020 at 03:02:59AM +0900, Fujii Masao wrote:
>>>>
>>>>
>>>> On 2020/04/02 9:41, Tomas Vondra wrote:
>>>>> Hi,
>>>>>
>>>>> I've pushed this after some minor cleanup and improvements.
>>>>
>>>> +static char *slru_names[] = {"async", "clog", "commit_timestamp",
>>>> +                              "multixact_offset", "multixact_member",
>>>> +                              "oldserxid", "pg_xact", "subtrans",
>>>> +                              "other" /* has to be last */};
>>>>
>>>> When I tried pg_stat_slru, I found that it returns a row for "pg_xact".
>>>> But since there is no "pg_xact" slru ("clog" slru exists instead),
>>>> "pg_xact" should be removed? Patch attached.
>>>>
>>>
>>> Yeah, I think I got confused and accidentally added both "clog" and
>>> "pg_xact". I'll get "pg_xact" removed.
>>
>> Thanks!
>>
> 
> OK, pushed. Thanks!

Thanks a lot!

But, like the patch that I attached in the previous email does,
"pg_xact" should be removed from the description of pg_stat_reset_slru()
in monitoring.sgml.

>> Another thing I found is; pgstat_send_slru() should be called also by
>> other processes than backend? For example, since clog data is flushed
>> basically by checkpointer, checkpointer seems to need to send slru stats.
>> Otherwise, pg_stat_slru.flushes would not be updated.
>>
> 
> Hmmm, that's a good point. If I understand the issue correctly, the
> checkpointer accumulates the stats but never really sends them because
> it never calls pgstat_report_stat/pgstat_send_slru. That's only called
> from PostgresMain, but not from CheckpointerMain.

Yes.

> I think we could simply add pgstat_send_slru() right after the existing
> call in CheckpointerMain, right?

Checkpointer sends off activity statistics to the stats collector in
two places, by calling pgstat_send_bgwriter(). What about calling
pgstat_send_slru() just after pgstat_send_bgwriter()?

In previous email, I mentioned checkpointer just as an example.
So probably we need to investigate what process should send slru stats,
other than checkpointer. I guess that at least autovacuum worker,
logical replication walsender and parallel query worker (maybe this has
been already covered by parallel query some mechanisms. Sorry I've
not checked that) would need to send its slru stats.

Atsushi-san reported another issue in pg_stat_slru.
You're planning to work on that?
https://postgr.es/m/CACZ0uYFe16pjZxQYaTn53mspyM7dgMPYL3DJLjjPw69GMCC2Ow@mail.gmail.com

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION