Horton

Thread

Commits

  1. doc: Improve descriptions of tup_returned and tup_fetched in pg_stat_database

  1. more detailed description of tup_returned and tup_fetched

    Masahiro Ikeda <ikedamsh@oss.nttdata.com> — 2021-05-14T08:00:45Z

    Hi,

I worried the difference between "tup_returned" and "tup_fetched" in
pg_stat_database. I assumed that "tup_returned" means the number of tuples
that returned to clients. Of course, this is wrong.

So, why don't you describe in more detail? If my understanding is right, they
correspond to "seq_tup_read" and "idx_tup_fetch" in pg_stat_all_tables.

Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION

  2. Re: more detailed description of tup_returned and tup_fetched

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2021-05-17T06:32:38Z

    
On 2021/05/14 17:00, Masahiro Ikeda wrote:
> Hi,
> 
> I worried the difference between "tup_returned" and "tup_fetched" in
> pg_stat_database. I assumed that "tup_returned" means the number of tuples
> that returned to clients. Of course, this is wrong.

-       Number of rows returned by queries in this database
+       Number of live rows returned by sequential scans of queries in this database

-       Number of rows fetched by queries in this database
+       Number of live rows fetched by index scan of queries in this database

I found the following comments in pgstat.h. So maybe even these
new descriptions are incorrect?

  * Note: for a table, tuples_returned is the number of tuples successfully
  * fetched by heap_getnext, while tuples_fetched is the number of tuples
  * successfully fetched by heap_fetch under the control of bitmap indexscans.
  * For an index, tuples_returned is the number of index entries returned by
  * the index AM, while tuples_fetched is the number of tuples successfully
  * fetched by heap_fetch under the control of simple indexscans for this index.

Regards,


-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

  3. Re: more detailed description of tup_returned and tup_fetched

    Masahiro Ikeda <ikedamsh@oss.nttdata.com> — 2021-05-17T09:58:17Z

    
On 2021/05/17 15:32, Fujii Masao wrote:
> 
> 
> On 2021/05/14 17:00, Masahiro Ikeda wrote:
>> Hi,
>>
>> I worried the difference between "tup_returned" and "tup_fetched" in
>> pg_stat_database. I assumed that "tup_returned" means the number of tuples
>> that returned to clients. Of course, this is wrong.
> 
> -       Number of rows returned by queries in this database
> +       Number of live rows returned by sequential scans of queries in this
> database
> 
> -       Number of rows fetched by queries in this database
> +       Number of live rows fetched by index scan of queries in this database
> 
> I found the following comments in pgstat.h. So maybe even these
> new descriptions are incorrect?
> 
>  * Note: for a table, tuples_returned is the number of tuples successfully
>  * fetched by heap_getnext, while tuples_fetched is the number of tuples
>  * successfully fetched by heap_fetch under the control of bitmap indexscans.
>  * For an index, tuples_returned is the number of index entries returned by
>  * the index AM, while tuples_fetched is the number of tuples successfully
>  * fetched by heap_fetch under the control of simple indexscans for this index.

Oh, Thanks!

I updated the sentences using the descriptions of
"pg_stat_all_tables.seq_tup_read", "pg_stat_all_tables.idx_tup_fetch", and
"pg_stat_all_index.idx_tup_read".

-       Number of rows returned by queries in this database
+       Number of rows returned by queries in this database. The rows
correspond to the live rows fetched by sequential scans and index entries
returned by scans on indexes

-       Number of rows fetched by queries in this database
+       Number of rows fetched by queries in this database. The rows
correspond to the live rows fetched by index scans

Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION

  4. Re: more detailed description of tup_returned and tup_fetched

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2021-05-17T11:46:38Z

    
On 2021/05/17 18:58, Masahiro Ikeda wrote:
> 
> 
> On 2021/05/17 15:32, Fujii Masao wrote:
>>
>>
>> On 2021/05/14 17:00, Masahiro Ikeda wrote:
>>> Hi,
>>>
>>> I worried the difference between "tup_returned" and "tup_fetched" in
>>> pg_stat_database. I assumed that "tup_returned" means the number of tuples
>>> that returned to clients. Of course, this is wrong.
>>
>> -       Number of rows returned by queries in this database
>> +       Number of live rows returned by sequential scans of queries in this
>> database
>>
>> -       Number of rows fetched by queries in this database
>> +       Number of live rows fetched by index scan of queries in this database
>>
>> I found the following comments in pgstat.h. So maybe even these
>> new descriptions are incorrect?
>>
>>   * Note: for a table, tuples_returned is the number of tuples successfully
>>   * fetched by heap_getnext, while tuples_fetched is the number of tuples
>>   * successfully fetched by heap_fetch under the control of bitmap indexscans.
>>   * For an index, tuples_returned is the number of index entries returned by
>>   * the index AM, while tuples_fetched is the number of tuples successfully
>>   * fetched by heap_fetch under the control of simple indexscans for this index.
> 
> Oh, Thanks!
> 
> I updated the sentences using the descriptions of
> "pg_stat_all_tables.seq_tup_read", "pg_stat_all_tables.idx_tup_fetch", and
> "pg_stat_all_index.idx_tup_read".
> 
> -       Number of rows returned by queries in this database
> +       Number of rows returned by queries in this database. The rows
> correspond to the live rows fetched by sequential scans and index entries
> returned by scans on indexes

This is still not correct because this counter is incremented even when
other scan like TidScan happens?

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

  5. Re: more detailed description of tup_returned and tup_fetched

    Masahiro Ikeda <ikedamsh@oss.nttdata.com> — 2021-05-18T04:20:59Z

    
On 2021/05/17 20:46, Fujii Masao wrote:
> 
> 
> On 2021/05/17 18:58, Masahiro Ikeda wrote:
>>
>>
>> On 2021/05/17 15:32, Fujii Masao wrote:
>>>
>>>
>>> On 2021/05/14 17:00, Masahiro Ikeda wrote:
>>>> Hi,
>>>>
>>>> I worried the difference between "tup_returned" and "tup_fetched" in
>>>> pg_stat_database. I assumed that "tup_returned" means the number of tuples
>>>> that returned to clients. Of course, this is wrong.
>>>
>>> -       Number of rows returned by queries in this database
>>> +       Number of live rows returned by sequential scans of queries in this
>>> database
>>>
>>> -       Number of rows fetched by queries in this database
>>> +       Number of live rows fetched by index scan of queries in this database
>>>
>>> I found the following comments in pgstat.h. So maybe even these
>>> new descriptions are incorrect?
>>>
>>>   * Note: for a table, tuples_returned is the number of tuples successfully
>>>   * fetched by heap_getnext, while tuples_fetched is the number of tuples
>>>   * successfully fetched by heap_fetch under the control of bitmap indexscans.
>>>   * For an index, tuples_returned is the number of index entries returned by
>>>   * the index AM, while tuples_fetched is the number of tuples successfully
>>>   * fetched by heap_fetch under the control of simple indexscans for this
>>> index.
>>
>> Oh, Thanks!
>>
>> I updated the sentences using the descriptions of
>> "pg_stat_all_tables.seq_tup_read", "pg_stat_all_tables.idx_tup_fetch", and
>> "pg_stat_all_index.idx_tup_read".
>>
>> -       Number of rows returned by queries in this database
>> +       Number of rows returned by queries in this database. The rows
>> correspond to the live rows fetched by sequential scans and index entries
>> returned by scans on indexes
> 
> This is still not correct because this counter is incremented even when
> other scan like TidScan happens?

Sorry, I couldn't find the way to increment tup_returned by TidScan.
Do you mean that Tid Range Scan increments the counter?

Tid Range Scan increments the tup_returned, and
pg_stat_all_tables.seq_tup_read is also incremented. I thought it's ok because
Tid Range Scan is like sequential scan. That's the reason why the document of
pg_stat_all_tables.seq_tup_read says "Number of live rows fetched by
sequential scans"

Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION

  6. Re: more detailed description of tup_returned and tup_fetched

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2021-05-18T07:01:47Z

    
On 2021/05/18 13:20, Masahiro Ikeda wrote:
> 
> 
> On 2021/05/17 20:46, Fujii Masao wrote:
>>
>>
>> On 2021/05/17 18:58, Masahiro Ikeda wrote:
>>>
>>>
>>> On 2021/05/17 15:32, Fujii Masao wrote:
>>>>
>>>>
>>>> On 2021/05/14 17:00, Masahiro Ikeda wrote:
>>>>> Hi,
>>>>>
>>>>> I worried the difference between "tup_returned" and "tup_fetched" in
>>>>> pg_stat_database. I assumed that "tup_returned" means the number of tuples
>>>>> that returned to clients. Of course, this is wrong.
>>>>
>>>> -       Number of rows returned by queries in this database
>>>> +       Number of live rows returned by sequential scans of queries in this
>>>> database
>>>>
>>>> -       Number of rows fetched by queries in this database
>>>> +       Number of live rows fetched by index scan of queries in this database
>>>>
>>>> I found the following comments in pgstat.h. So maybe even these
>>>> new descriptions are incorrect?
>>>>
>>>>    * Note: for a table, tuples_returned is the number of tuples successfully
>>>>    * fetched by heap_getnext, while tuples_fetched is the number of tuples
>>>>    * successfully fetched by heap_fetch under the control of bitmap indexscans.
>>>>    * For an index, tuples_returned is the number of index entries returned by
>>>>    * the index AM, while tuples_fetched is the number of tuples successfully
>>>>    * fetched by heap_fetch under the control of simple indexscans for this
>>>> index.
>>>
>>> Oh, Thanks!
>>>
>>> I updated the sentences using the descriptions of
>>> "pg_stat_all_tables.seq_tup_read", "pg_stat_all_tables.idx_tup_fetch", and
>>> "pg_stat_all_index.idx_tup_read".
>>>
>>> -       Number of rows returned by queries in this database
>>> +       Number of rows returned by queries in this database. The rows
>>> correspond to the live rows fetched by sequential scans and index entries
>>> returned by scans on indexes
>>
>> This is still not correct because this counter is incremented even when
>> other scan like TidScan happens?
> 
> Sorry, I couldn't find the way to increment tup_returned by TidScan.
> Do you mean that Tid Range Scan increments the counter?

Yes, what I tried to mean is Tid Range Scan.

> 
> Tid Range Scan increments the tup_returned, and
> pg_stat_all_tables.seq_tup_read is also incremented. I thought it's ok because
> Tid Range Scan is like sequential scan.

Yes, you're right. One interesting thing I found is;
when Tid Range Scan happens, seq_tup_read is incremented
but seq_scan is not. I'm not sure if this is expected behavior or not.

> That's the reason why the document of
> pg_stat_all_tables.seq_tup_read says "Number of live rows fetched by
> sequential scans"

Regarding the original issue, as far as I understand correctly,

* pg_stat_database.tup_returned = sum(pg_stat_all_tables.seq_tup_read) + sum(pg_stat_all_indexes.idx_tup_read)
* pg_stat_database.tup_fetched = sum(pg_stat_all_tables.idx_tup_fetch)

But the counters for some system catalogs like pg_database shared
across all databases of a cluster are excluded from that calculation.
Is this my understanding right? If right, probably we can reuse
the existing descriptions for those counters to document
pg_stat_database counters. For example,

pg_stat_database.tup_returned:
Number of live rows fetched by sequential and index scans in this database

pg_stat_database.tup_fetched:
Number of index entries returned by scans on indexes in this database

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

  7. Re: more detailed description of tup_returned and tup_fetched

    Masahiro Ikeda <ikedamsh@oss.nttdata.com> — 2021-05-18T09:23:30Z

    
On 2021/05/18 16:01, Fujii Masao wrote:
> On 2021/05/18 13:20, Masahiro Ikeda wrote:
>> Tid Range Scan increments the tup_returned, and
>> pg_stat_all_tables.seq_tup_read is also incremented. I thought it's ok because
>> Tid Range Scan is like sequential scan.
> 
> Yes, you're right. One interesting thing I found is;
> when Tid Range Scan happens, seq_tup_read is incremented
> but seq_scan is not. I'm not sure if this is expected behavior or not.

The following comment says that this behavior is expected. But, I agree it's
odd and it's natural both seq_tup_read and seq_scan are incremented at the
same time or not...

/*
 * Currently, we only have a stats counter for sequential heap scans (but
 * e.g for bitmap scans the underlying bitmap index scans will be counted,
 * and for sample scans we update stats for tuple fetches).
 */
if (scan->rs_base.rs_flags & SO_TYPE_SEQSCAN)
	pgstat_count_heap_scan(scan->rs_base.rs_rd);


>> That's the reason why the document of
>> pg_stat_all_tables.seq_tup_read says "Number of live rows fetched by
>> sequential scans"
> 
> Regarding the original issue, as far as I understand correctly,
> 
> * pg_stat_database.tup_returned = sum(pg_stat_all_tables.seq_tup_read) +
> sum(pg_stat_all_indexes.idx_tup_read)
> * pg_stat_database.tup_fetched = sum(pg_stat_all_tables.idx_tup_fetch)
> 
> But the counters for some system catalogs like pg_database shared
> across all databases of a cluster are excluded from that calculation.
> Is this my understanding right? If right, probably we can reuse
> the existing descriptions for those counters to document
> pg_stat_database counters. For example,

Yes, my understanding is same now.


> pg_stat_database.tup_returned:> Number of live rows fetched by sequential and index scans in this database

I wonder "live rows fetched by index scans" may mislead. I think "live" means
it's not dead tuple and "rows" mean the tuple user want to get.

But, pg_stat_all_indexes.idx_tup_read says that "index entires returned by
scans on this index". There is no meaning of "live" and "rows", so I thought
it's better to distinguish them.

So, why don't you change to "Number of live rows fetched by sequential scans
and index entries returned by index scans in this database"?


> pg_stat_database.tup_fetched:
> Number of index entries returned by scans on indexes in this database
Is this the sum of pg_stat_all_indexes.idx_tup_read? This is accounted to
pg_stat_database.tup_returned.

"Number of live rows fetched by index scans in this database" seems to be correct.


Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION

  8. Re: more detailed description of tup_returned and tup_fetched

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2021-05-18T11:10:56Z

    
On 2021/05/18 18:23, Masahiro Ikeda wrote:
> 
> 
> On 2021/05/18 16:01, Fujii Masao wrote:
>> On 2021/05/18 13:20, Masahiro Ikeda wrote:
>>> Tid Range Scan increments the tup_returned, and
>>> pg_stat_all_tables.seq_tup_read is also incremented. I thought it's ok because
>>> Tid Range Scan is like sequential scan.
>>
>> Yes, you're right. One interesting thing I found is;
>> when Tid Range Scan happens, seq_tup_read is incremented
>> but seq_scan is not. I'm not sure if this is expected behavior or not.
> 
> The following comment says that this behavior is expected. But, I agree it's
> odd and it's natural both seq_tup_read and seq_scan are incremented at the
> same time or not...
> 
> /*
>   * Currently, we only have a stats counter for sequential heap scans (but
>   * e.g for bitmap scans the underlying bitmap index scans will be counted,
>   * and for sample scans we update stats for tuple fetches).
>   */
> if (scan->rs_base.rs_flags & SO_TYPE_SEQSCAN)
> 	pgstat_count_heap_scan(scan->rs_base.rs_rd);
> 
> 
>>> That's the reason why the document of
>>> pg_stat_all_tables.seq_tup_read says "Number of live rows fetched by
>>> sequential scans"
>>
>> Regarding the original issue, as far as I understand correctly,
>>
>> * pg_stat_database.tup_returned = sum(pg_stat_all_tables.seq_tup_read) +
>> sum(pg_stat_all_indexes.idx_tup_read)
>> * pg_stat_database.tup_fetched = sum(pg_stat_all_tables.idx_tup_fetch)
>>
>> But the counters for some system catalogs like pg_database shared
>> across all databases of a cluster are excluded from that calculation.
>> Is this my understanding right? If right, probably we can reuse
>> the existing descriptions for those counters to document
>> pg_stat_database counters. For example,
> 
> Yes, my understanding is same now.
> 
> 
>> pg_stat_database.tup_returned:> Number of live rows fetched by sequential and index scans in this database
> 
> I wonder "live rows fetched by index scans" may mislead. I think "live" means
> it's not dead tuple and "rows" mean the tuple user want to get.
> 
> But, pg_stat_all_indexes.idx_tup_read says that "index entires returned by
> scans on this index". There is no meaning of "live" and "rows", so I thought
> it's better to distinguish them.
> 
> So, why don't you change to "Number of live rows fetched by sequential scans
> and index entries returned by index scans in this database"?

Yes, LGTM.


>> pg_stat_database.tup_fetched:
>> Number of index entries returned by scans on indexes in this database
> Is this the sum of pg_stat_all_indexes.idx_tup_read? This is accounted to
> pg_stat_database.tup_returned.

I was thinking that pg_stat_database.tup_fetched is the same as
the sum of pg_stat_all_tables.idx_tup_fetch. Because they both
are incremented by bitmap index scans, but pg_stat_all_indexes.idx_tup_read
is not.

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

  9. Re: more detailed description of tup_returned and tup_fetched

    Masahiro Ikeda <ikedamsh@oss.nttdata.com> — 2021-05-20T00:46:50Z

    
On 2021/05/18 20:10, Fujii Masao wrote:
> 
> 
> On 2021/05/18 18:23, Masahiro Ikeda wrote:
>>
>>
>> On 2021/05/18 16:01, Fujii Masao wrote:
>>> On 2021/05/18 13:20, Masahiro Ikeda wrote:
>>>> Tid Range Scan increments the tup_returned, and
>>>> pg_stat_all_tables.seq_tup_read is also incremented. I thought it's ok
>>>> because
>>>> Tid Range Scan is like sequential scan.
>>>
>>> Yes, you're right. One interesting thing I found is;
>>> when Tid Range Scan happens, seq_tup_read is incremented
>>> but seq_scan is not. I'm not sure if this is expected behavior or not.
>>
>> The following comment says that this behavior is expected. But, I agree it's
>> odd and it's natural both seq_tup_read and seq_scan are incremented at the
>> same time or not...
>>
>> /*
>>   * Currently, we only have a stats counter for sequential heap scans (but
>>   * e.g for bitmap scans the underlying bitmap index scans will be counted,
>>   * and for sample scans we update stats for tuple fetches).
>>   */
>> if (scan->rs_base.rs_flags & SO_TYPE_SEQSCAN)
>>     pgstat_count_heap_scan(scan->rs_base.rs_rd);
>>
>>
>>>> That's the reason why the document of
>>>> pg_stat_all_tables.seq_tup_read says "Number of live rows fetched by
>>>> sequential scans"
>>>
>>> Regarding the original issue, as far as I understand correctly,
>>>
>>> * pg_stat_database.tup_returned = sum(pg_stat_all_tables.seq_tup_read) +
>>> sum(pg_stat_all_indexes.idx_tup_read)
>>> * pg_stat_database.tup_fetched = sum(pg_stat_all_tables.idx_tup_fetch)
>>>
>>> But the counters for some system catalogs like pg_database shared
>>> across all databases of a cluster are excluded from that calculation.
>>> Is this my understanding right? If right, probably we can reuse
>>> the existing descriptions for those counters to document
>>> pg_stat_database counters. For example,
>>
>> Yes, my understanding is same now.
>>
>>
>>> pg_stat_database.tup_returned:> Number of live rows fetched by sequential
>>> and index scans in this database
>>
>> I wonder "live rows fetched by index scans" may mislead. I think "live" means
>> it's not dead tuple and "rows" mean the tuple user want to get.
>>
>> But, pg_stat_all_indexes.idx_tup_read says that "index entires returned by
>> scans on this index". There is no meaning of "live" and "rows", so I thought
>> it's better to distinguish them.
>>
>> So, why don't you change to "Number of live rows fetched by sequential scans
>> and index entries returned by index scans in this database"?
> 
> Yes, LGTM.
> 
> 
>>> pg_stat_database.tup_fetched:
>>> Number of index entries returned by scans on indexes in this database
>> Is this the sum of pg_stat_all_indexes.idx_tup_read? This is accounted to
>> pg_stat_database.tup_returned.
> 
> I was thinking that pg_stat_database.tup_fetched is the same as
> the sum of pg_stat_all_tables.idx_tup_fetch. Because they both
> are incremented by bitmap index scans, but pg_stat_all_indexes.idx_tup_read
> is not.

Yes. So, "Number of index entries returned by scans on indexes in this
database" is incorrect, and "Number of live rows fetched by index scans in
this database" is correct?

Regards,

-- 
Masahiro Ikeda
NTT DATA CORPORATION

  10. Re: more detailed description of tup_returned and tup_fetched

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2021-05-20T08:00:13Z

    
On 2021/05/20 9:46, Masahiro Ikeda wrote:
> 
> 
> On 2021/05/18 20:10, Fujii Masao wrote:
>>
>>
>> On 2021/05/18 18:23, Masahiro Ikeda wrote:
>>>
>>>
>>> On 2021/05/18 16:01, Fujii Masao wrote:
>>>> On 2021/05/18 13:20, Masahiro Ikeda wrote:
>>>>> Tid Range Scan increments the tup_returned, and
>>>>> pg_stat_all_tables.seq_tup_read is also incremented. I thought it's ok
>>>>> because
>>>>> Tid Range Scan is like sequential scan.
>>>>
>>>> Yes, you're right. One interesting thing I found is;
>>>> when Tid Range Scan happens, seq_tup_read is incremented
>>>> but seq_scan is not. I'm not sure if this is expected behavior or not.
>>>
>>> The following comment says that this behavior is expected. But, I agree it's
>>> odd and it's natural both seq_tup_read and seq_scan are incremented at the
>>> same time or not...
>>>
>>> /*
>>>    * Currently, we only have a stats counter for sequential heap scans (but
>>>    * e.g for bitmap scans the underlying bitmap index scans will be counted,
>>>    * and for sample scans we update stats for tuple fetches).
>>>    */
>>> if (scan->rs_base.rs_flags & SO_TYPE_SEQSCAN)
>>>      pgstat_count_heap_scan(scan->rs_base.rs_rd);
>>>
>>>
>>>>> That's the reason why the document of
>>>>> pg_stat_all_tables.seq_tup_read says "Number of live rows fetched by
>>>>> sequential scans"
>>>>
>>>> Regarding the original issue, as far as I understand correctly,
>>>>
>>>> * pg_stat_database.tup_returned = sum(pg_stat_all_tables.seq_tup_read) +
>>>> sum(pg_stat_all_indexes.idx_tup_read)
>>>> * pg_stat_database.tup_fetched = sum(pg_stat_all_tables.idx_tup_fetch)
>>>>
>>>> But the counters for some system catalogs like pg_database shared
>>>> across all databases of a cluster are excluded from that calculation.
>>>> Is this my understanding right? If right, probably we can reuse
>>>> the existing descriptions for those counters to document
>>>> pg_stat_database counters. For example,
>>>
>>> Yes, my understanding is same now.
>>>
>>>
>>>> pg_stat_database.tup_returned:> Number of live rows fetched by sequential
>>>> and index scans in this database
>>>
>>> I wonder "live rows fetched by index scans" may mislead. I think "live" means
>>> it's not dead tuple and "rows" mean the tuple user want to get.
>>>
>>> But, pg_stat_all_indexes.idx_tup_read says that "index entires returned by
>>> scans on this index". There is no meaning of "live" and "rows", so I thought
>>> it's better to distinguish them.
>>>
>>> So, why don't you change to "Number of live rows fetched by sequential scans
>>> and index entries returned by index scans in this database"?
>>
>> Yes, LGTM.
>>
>>
>>>> pg_stat_database.tup_fetched:
>>>> Number of index entries returned by scans on indexes in this database
>>> Is this the sum of pg_stat_all_indexes.idx_tup_read? This is accounted to
>>> pg_stat_database.tup_returned.
>>
>> I was thinking that pg_stat_database.tup_fetched is the same as
>> the sum of pg_stat_all_tables.idx_tup_fetch. Because they both
>> are incremented by bitmap index scans, but pg_stat_all_indexes.idx_tup_read
>> is not.
> 
> Yes. So, "Number of index entries returned by scans on indexes in this
> database" is incorrect, and "Number of live rows fetched by index scans in
> this database" is correct?

Yes, I think so!

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

  11. Re: more detailed description of tup_returned and tup_fetched

    Masahiro Ikeda <ikedamsh@oss.nttdata.com> — 2021-05-20T08:38:44Z

    
On 2021/05/20 17:00, Fujii Masao wrote:
> On 2021/05/20 9:46, Masahiro Ikeda wrote:
>> On 2021/05/18 20:10, Fujii Masao wrote:
>>>>> pg_stat_database.tup_fetched:
>>>>> Number of index entries returned by scans on indexes in this database
>>>> Is this the sum of pg_stat_all_indexes.idx_tup_read? This is accounted to
>>>> pg_stat_database.tup_returned.
>>>
>>> I was thinking that pg_stat_database.tup_fetched is the same as
>>> the sum of pg_stat_all_tables.idx_tup_fetch. Because they both
>>> are incremented by bitmap index scans, but pg_stat_all_indexes.idx_tup_read
>>> is not.
>>
>> Yes. So, "Number of index entries returned by scans on indexes in this
>> database" is incorrect, and "Number of live rows fetched by index scans in
>> this database" is correct?
> 
> Yes, I think so!

Thanks!
I updated the patch for summarizing this thread.

Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION

  12. Re: more detailed description of tup_returned and tup_fetched

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2021-05-21T13:26:14Z

    
On 2021/05/20 17:38, Masahiro Ikeda wrote:
> 
> 
> On 2021/05/20 17:00, Fujii Masao wrote:
>> On 2021/05/20 9:46, Masahiro Ikeda wrote:
>>> On 2021/05/18 20:10, Fujii Masao wrote:
>>>>>> pg_stat_database.tup_fetched:
>>>>>> Number of index entries returned by scans on indexes in this database
>>>>> Is this the sum of pg_stat_all_indexes.idx_tup_read? This is accounted to
>>>>> pg_stat_database.tup_returned.
>>>>
>>>> I was thinking that pg_stat_database.tup_fetched is the same as
>>>> the sum of pg_stat_all_tables.idx_tup_fetch. Because they both
>>>> are incremented by bitmap index scans, but pg_stat_all_indexes.idx_tup_read
>>>> is not.
>>>
>>> Yes. So, "Number of index entries returned by scans on indexes in this
>>> database" is incorrect, and "Number of live rows fetched by index scans in
>>> this database" is correct?
>>
>> Yes, I think so!
> 
> Thanks!
> I updated the patch for summarizing this thread.

Thanks for updating the patch! LGTM.

This is an improvement of documentation, so this should be applied in
v15 dev cycle? If so, could you add the patch to the next CF? Or you think
this is a bug fix and needs to be back-patched?

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

  13. Re: more detailed description of tup_returned and tup_fetched

    Masahiro Ikeda <ikedamsh@oss.nttdata.com> — 2021-05-24T02:37:09Z

    
On 2021/05/21 22:26, Fujii Masao wrote:
> 
> 
> On 2021/05/20 17:38, Masahiro Ikeda wrote:
>>
>>
>> On 2021/05/20 17:00, Fujii Masao wrote:
>>> On 2021/05/20 9:46, Masahiro Ikeda wrote:
>>>> On 2021/05/18 20:10, Fujii Masao wrote:
>>>>>>> pg_stat_database.tup_fetched:
>>>>>>> Number of index entries returned by scans on indexes in this database
>>>>>> Is this the sum of pg_stat_all_indexes.idx_tup_read? This is accounted to
>>>>>> pg_stat_database.tup_returned.
>>>>>
>>>>> I was thinking that pg_stat_database.tup_fetched is the same as
>>>>> the sum of pg_stat_all_tables.idx_tup_fetch. Because they both
>>>>> are incremented by bitmap index scans, but pg_stat_all_indexes.idx_tup_read
>>>>> is not.
>>>>
>>>> Yes. So, "Number of index entries returned by scans on indexes in this
>>>> database" is incorrect, and "Number of live rows fetched by index scans in
>>>> this database" is correct?
>>>
>>> Yes, I think so!
>>
>> Thanks!
>> I updated the patch for summarizing this thread.
> 
> Thanks for updating the patch! LGTM.
> 
> This is an improvement of documentation, so this should be applied in
> v15 dev cycle? If so, could you add the patch to the next CF? Or you think
> this is a bug fix and needs to be back-patched?

Thanks for checking the patch!
I thought this patch will be applied in V15 dev cycle.

OK. I added the patch to the next CF.
https://commitfest.postgresql.org/33/3130/

Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION

  14. Re: more detailed description of tup_returned and tup_fetched

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2021-06-30T11:59:41Z

    
On 2021/05/24 11:37, Masahiro Ikeda wrote:
> Thanks for checking the patch!
> I thought this patch will be applied in V15 dev cycle.
> 
> OK. I added the patch to the next CF.
> https://commitfest.postgresql.org/33/3130/

I pushed the patch to 15dev. Thanks!

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

  15. Re: more detailed description of tup_returned and tup_fetched

    Masahiro Ikeda <ikedamsh@oss.nttdata.com> — 2021-07-01T02:34:52Z

    
On 2021/06/30 20:59, Fujii Masao wrote:
> 
> 
> On 2021/05/24 11:37, Masahiro Ikeda wrote:
>> Thanks for checking the patch!
>> I thought this patch will be applied in V15 dev cycle.
>>
>> OK. I added the patch to the next CF.
>> https://commitfest.postgresql.org/33/3130/
> 
> I pushed the patch to 15dev. Thanks!

Thanks!

Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION