Re: Add index scan progress to pg_stat_progress_vacuum

>    Indeed.

>    It might have already been discussed but other than using a new shmem
>    hash for parallel vacuum, I wonder if we can allow workers to change
>    the leader’s progress information. It would break the assumption that
>    the backend status entry is modified by its own backend, though. But
>    it might help for progress updates of other parallel operations too.
>    This essentially does the same thing as what the current patch does
>    but it doesn't require a new shmem hash.

I experimented with this idea, but it did not work. The idea would have been to create a pgstat_progress_update function that takes the leader pid, however infrastructure does not exist to allow one backend to manipulate another backends backend status array.
pgstat_fetch_stat_beentry returns a local copy only. 

>    Another idea I come up with is that the parallel vacuum leader checks
>    PVIndStats.status and updates how many indexes are processed to its
>    progress information. The leader can check it and update the progress
>    information before and after index vacuuming. And possibly we can add
>    a callback to the main loop of index AM's bulkdelete and vacuumcleanup
>    so that the leader can periodically make it up-to-date.

>    Regards,

The PVIndStats idea is also one I experimented with but it did not work. The reason being the backend checking the progress needs to do a shm_toc_lookup to access the data, but they are not prepared to do so. 

I have not considered the callback in the index AM's bulkdelete and vacuumcleanup, but I can imagine this is not possible since a leader could be busy vacuuming rather than updating counters, but I may be misunderstanding the suggestion.

--
Sami Imseih
Amazon Web Services