Horton

Re: SLRU statistics

Tomas Vondra <tomas.vondra@2ndquadrant.com>

View thread
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2020-01-20T18:45:46Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Fix async.c to not register any SLRU stats counts in the postmaster.

  2. Use proper GetDatum function in pg_stat_get_slru().

  3. Initialize SLRU stats entries to zero.

  4. Remove pg_xact from pg_stat_reset_slru docs

  5. Remove pg_xact entry from SLRU stats

  6. Track SLRU page hits in SimpleLruReadPage_ReadOnly

  7. Fix typo in SLRU stats documentation

  8. Collect statistics about SLRU caches

  9. Simplify and rename some GUC variables, per various recent discussions:

On Mon, Jan 20, 2020 at 03:01:36PM -0300, Alvaro Herrera wrote:
>On 2020-Jan-20, Tomas Vondra wrote:
>
>> On Mon, Jan 20, 2020 at 01:04:33AM +0000, tsunakawa.takay@fujitsu.com wrote:
>> > From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
>> > > One of the stats I occasionally wanted to know are stats for the SLRU
>> > > stats (we have couple of those - clog, subtrans, ...). So here is a WIP
>> > > version of a patch adding that.
>> >
>> > How can users take advantage of this information?  I think we also need
>> > the ability to set the size of SLRU buffers.  (I want to be freed from
>> > the concern about the buffer shortage by setting the buffer size to its
>> > maximum.  For example, CLOG would be only 1 GB.)
>>
>> You're right the users can't really take advantage of this - my primary
>> motivation was providing a feedback for devs, benchmarking etc. That
>> might have been done with DEBUG messages or something, but this seems
>> more convenient.
>
>I think the stats are definitely needed if we keep the current code.
>I've researched some specific problems in this code, such as the need
>for more subtrans SLRU buffers; IIRC it was pretty painful to figure out
>what the problem was without counters, and it'd have been trivial with
>them.
>

Right. Improving our ability to monitor/measure things is the goal of
this patch.

>> I think it's unclear how desirable / necessary it is to allow users to
>> tweak those caches. I don't think we should have a GUC for everything,
>> but maybe there's some sort of heuristics to determine the size. The
>> assumption is we actually find practical workloads where the size of
>> these SLRUs is a performance issue.
>
>I expect we'll eventually realize the need for changes in this area.
>Either configurability in the buffer pool sizes, or moving them to be
>part of shared_buffers (IIRC Thomas Munro had a patch for this.)
>Example: SLRUs like pg_commit and pg_subtrans have higher buffer
>consumption as the range of open transactions increases; for many users
>this is not a concern and they can live with the default values.
>
>(I think when pg_commit (née pg_clog) buffers were increased, we should
>have increased pg_subtrans buffers to match.)
>

Quite possibly, yes. All I'm saying is that it's not something I intend
to address with this patch. It's quite possible the solutions will be
different for each SLRU, and that will require more research.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services