CLUSTER command progress monitor
Tatsuro Yamada <yamada.tatsuro@lab.ntt.co.jp>
From: Tatsuro Yamada <yamada.tatsuro@lab.ntt.co.jp>
To: pgsql-hackers@postgresql.org
Date: 2017-08-31T02:12:02Z
Lists: pgsql-hackers
Attachments
- progress_monitor_cluster_v1.patch (text/x-patch) patch v1
Hi,
Following is a proposal for reporting the progress of CLUSTER command:
It seems that the following could be the phases of CLUSTER processing:
1. scanning heap
2. sort tuples
3. writing new heap
4. scan heap and write new heap
5. swapping relation files
6. rebuild index
7. performing final cleanup
These phases are based on Rahila's presentation at PGCon 2017
(https://www.pgcon.org/2017/schedule/attachments/472_Progress%20Measurement%20PostgreSQL.pdf)
and I added some phases on it.
CLUSTER command may use Index Scan or Seq Scan when scanning the heap.
Depending on which one is chosen, the command will proceed in the
following sequence of phases:
* Seq Scan
1. scanning heap
2. sort tuples
3. writing new heap
5. swapping relation files
6. rebuild index
7. performing final cleanup
* Index Scan
4. scan heap and write new heap
5. swapping relation files
6. rebuild index
7. performing final cleanup
The view provides the information of CLUSTER command progress details as follows
postgres=# \d pg_stat_progress_cluster
View "pg_catalog.pg_stat_progress_cluster"
Column | Type | Collation | Nullable | Default
---------------------+---------+-----------+----------+---------
pid | integer | | |
datid | oid | | |
datname | name | | |
relid | oid | | |
phase | text | | |
scan_method | text | | |
scan_index_relid | bigint | | |
heap_tuples_total | bigint | | |
heap_tuples_scanned | bigint | | |
Then I have questions.
* Should we have separate views for them? Or should both be covered by the
same view with some indication of which command (CLUSTER or VACUUM FULL)
is actually running?
I mean this progress monitor could be covering not only CLUSTER command but also
VACUUM FULL command.
* I chose tuples as scan heap's counter (heap_tuples_scanned) since it's not
easy to get current blocks from Index Scan. Is it Ok?
I'll add this patch to CF2017-09.
Any comments or suggestion are welcome.
Regards,
Tatsuro Yamada
NTT Open Source Software Center
Commits
-
Fix progress reporting of CLUSTER / VACUUM FULL
- 6212276e4343 13.0 landed
- da47e43dc32e 12.0 landed
-
Fix thinko when ending progress report for a backend
- f4b91a50e9e6 9.6.16 landed
- 4950f09e7ae2 10.11 landed
- af28744288b1 11.6 landed
- 4c3e750ab4b5 12.0 landed
- ae060a52b288 13.0 landed
-
Report progress of REINDEX operations
- 03f9e5cba0ee 12.0 cited
-
Add progress reporting for CLUSTER and VACUUM FULL.
- 6f97457e0ddd 12.0 landed
-
Un-hide most cascaded-drop details in regression test results.
- 940311e4bb32 12.0 cited
-
Fix memory leak in printtup.c.
- f2004f19ed9c 12.0 cited
-
Remove unused macro
- fb5806533f9f 12.0 cited