Horton

Re: CLUSTER command progress monitor

Robert Haas <robertmhaas@gmail.com>

View thread View on postgr.es
From: Robert Haas <robertmhaas@gmail.com>
To: Tatsuro Yamada <yamada.tatsuro@lab.ntt.co.jp>
Cc: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2017-09-08T09:55:53Z
Lists: pgsql-hackers 
On Wed, Aug 30, 2017 at 10:12 PM, Tatsuro Yamada
<yamada.tatsuro@lab.ntt.co.jp> wrote:
>   1. scanning heap
>   2. sort tuples

These two phases overlap, though. I believe progress reporting for
sorts is really hard.  In the simple case where the data fits in
work_mem, none of the work of the sort gets done until all the data is
read.  Once you switch to an external sort, you're writing batch
files, so a lot of the work is now being done during data loading.
But as the number of batch files grows, the final merge at the end
becomes an increasingly noticeable part of the cost, and eventually
you end up needing multiple merge passes.  I think we need some smart
way to report on sorts so that we can tell how much of the work has
really been done, but I don't know how to do it.

>  heap_tuples_total   | bigint  |           |          |

The patch is getting the value reported as heap_tuples_total from
OldHeap->rd_rel->reltuples.  I think this is pointless: the user can
see that value anyway if they wish.  The point of the progress
counters is to expose things the user couldn't otherwise see.  It's
also not necessarily accurate: it's only an estimate in the best case,
and may be way off if the relation has recently be extended by a large
amount.  I think it's pretty important that we try hard to only report
values that are known to be accurate, because users hate (and mock)
inaccurate progress reports.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. Fix progress reporting of CLUSTER / VACUUM FULL

  2. Fix thinko when ending progress report for a backend

  3. Report progress of REINDEX operations

  4. Add progress reporting for CLUSTER and VACUUM FULL.

  5. Un-hide most cascaded-drop details in regression test results.

  6. Fix memory leak in printtup.c.

  7. Remove unused macro