Re: [HACKERS] CLUSTER command progress monitor

Tatsuro Yamada <yamatattsu@gmail.com>

From: Tattsu Yama <yamatattsu@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>, Tatsuro Yamada <tatsuro.yamada.tf@nttcom.co.jp>, robertmhaas@gmail.com, Rafia Sabih <rafia.pghackers@gmail.com>, Dmitry Dolgov <9erthalion6@gmail.com>, Peter Geoghegan <pg@bowt.ie>, pgsql-hackers@postgresql.org, Jeff Janes <jeff.janes@gmail.com>, Thomas Munro <thomas.munro@enterprisedb.com>, Michael Paquier <michael.paquier@gmail.com>
Date: 2019-09-15T03:35:10Z
Lists: pgsql-hackers
Hi Michael!

> Attached file is WIP patch.In my patch, I added "command id" to all APIs
> of
> > progress reporting to isolate commands. Therefore, it doesn't allow to
> > cascade updating system views. And my patch is on WIP so it needs
> clean-up
> > and test.
> > I share it anyway. :)
>
> +       if (cmdtype == PROGRESS_COMMAND_INVALID ||
> beentry->st_progress_command == cmdtype)
> +       {
> +               PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
> +               beentry->st_progress_param[index] = val;
> +               PGSTAT_END_WRITE_ACTIVITY(beentry);
> +       }
> You basically don't need the progress reports if the command ID is
> invalid, no?
>


Ah, right.
I'll check and fix that today. :)



>
> Another note is that you don't actually fix the problems related to
> the calls of pgstat_progress_end_command() which have been added for
> REINDEX reporting, so a progress report started for CLUSTER can get
> ended earlier than expected, preventing the follow-up progress updates
> to show up.
>
>

Hmm... I fixed the problem. Please confirm the test result repeated below.
CLUSTER is able to get the last phase: performing final clean up by using
the patch.

# Test result
========================================
postgres=# select * from pg_stat_progress_cluster ; \watch 0.001;
11636|13591|postgres|16384|CLUSTER|initializing|0|0|0|0|0|0
11636|13591|postgres|16384|CLUSTER|index scanning heap|16389|251|251|0|0|0
11636|13591|postgres|16384|CLUSTER|index scanning
heap|16389|10000|10000|0|0|0
11636|13591|postgres|16384|CLUSTER|rebuilding
index|16389|10000|10000|0|0|0  <== The last column rebuild_index_count is
increasing!
11636|13591|postgres|16384|CLUSTER|rebuilding index|16389|10000|10000|0|0|1
11636|13591|postgres|16384|CLUSTER|rebuilding index|16389|10000|10000|0|0|2
11636|13591|postgres|16384|CLUSTER|rebuilding index|16389|10000|10000|0|0|3
11636|13591|postgres|16384|CLUSTER|rebuilding index|16389|10000|10000|0|0|4
11636|13591|postgres|16384|CLUSTER|performing final
cleanup|16389|10000|10000|0|0|5  <== The last phase of CLUSTER!
========================================

Thanks,
Tatsuro Yamada

Commits

  1. Fix progress reporting of CLUSTER / VACUUM FULL