Re: Add index scan progress to pg_stat_progress_vacuum

>   I think that indexes_total should be 0 also when INDEX_CLEANUP is off.

Patch updated for handling of INDEX_CLEANUP = off, with an update to
the documentation as well.

>    I think we don't need to reset it at the end of index vacuuming. There
>    is a small window before switching to the next phase. If we reset this
>    value while showing "index vacuuming" phase, the user might get
>    confused. Instead, we can reset it at the beginning of the index
>    vacuuming.

No, I think the way it's currently done is correct. We want to reset the number
of indexes completed before we increase the num_index_scans ( index vacuum cycle ).
This ensures that when we enter a new index cycle, the number of indexes completed
are already reset. The 2 fields that matter here is how many indexes are vacuumed in the
currently cycle and this is called out in the documentation as such.

>    We should add CHECK_FOR_INTERRUPTS() at the beginning of the loop to
>    make the wait interruptible.

Done.

>    I think it would be better to update the counter only when the value
>    has been increased.

Done. Did so by checking the progress value from the beentry directly
in the function. We can do a more generalized 

>    I think we should set a timeout, say 1 sec, to WaitLatch so that it
>    can periodically check the progress.

Done.

>    Probably it's better to have a new wait event for this wait in order
>    to distinguish wait for workers to complete index vacuum from the wait
>    for workers to exit.

I was trying to avoid introducing a new wait event, but thinking about it, 
I agree with your suggestion. 

I created a new ParallelVacuumFinish wait event and documentation
for the wait event.


>    I think this doesn't work, since ivinfo.idx_completed is in the
>    backend-local memory. Instead, I think we can have a function in
>    vacuumparallel.c that updates the progress. Then we can have index AM
>    call this function.

Yeah, you're absolutely correct. 

To make this work correctly, I did something similar to VacuumActiveNWorkers
and introduced an extern variable called ParallelVacuumProgress.
This variable points to pvs->shared->idx_completed_progress. 

The index AMs then call parallel_vacuum_update_progress which
Is responsible for updating the progress with the current value
of ParallelVacuumProgress. 

ParallelVacuumProgress is also set to NULL at the end of every index cycle.

>   ivinfo.report_parallel_progress = !IsParallelWorker();

Done


Regards,

Sami Imseih
Amazon Web Services (AWS)