Re: monitoring CREATE INDEX [CONCURRENTLY]

Rahila Syed <rahila.syed@2ndquadrant.com>

From: Rahila Syed <rahila.syed@2ndquadrant.com>
To: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, Simon Riggs <simon@2ndquadrant.com>, Pg Hackers <pgsql-hackers@lists.postgresql.org>, David Fetter <david@fetter.org>, Pavan Deolasee <pavan.deolasee@gmail.com>, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
Date: 2019-03-24T20:42:28Z
Lists: pgsql-hackers
Hi Alvaro,

Please see few comments below:

1. Makecheck fails currently as view definition of expected rules.out does
not reflect latest changes in progress metrics numbering.

2. +      <entry>
+       When creating an index on a partitioned, this column is set to the
+       total number of partitions on which the index is to be created.
+      </entry>
+     </row>
+     <row>
+      <entry><structfield>partitions_done</structfield></entry>
+      <entry><type>bigint</type></entry>
+      <entry>
+       When creating an index on a partitioned, this column is set to the

I think there is a typo here 's/partitioned/partitioned table/'

3.
+       if (hscan->rs_base.rs_parallel != NULL)
+       {
+               ParallelBlockTableScanDesc bpscan;
+
+               bpscan = (ParallelBlockTableScanDesc)
hscan->rs_base.rs_parallel;
+               startblock = bpscan->phs_startblock;
+       }
+       else
+               startblock = hscan->rs_startblock;
+
+       /*
+        * Might have wrapped around the end of the relation, if startblock
was
+        * not zero.
+        */
+       if (hscan->rs_cblock > startblock)
+               blocks_done = hscan->rs_cblock - startblock;
+       else
+               blocks_done = hscan->rs_nblocks - startblock +
+                       hscan->rs_cblock;
+
+       return blocks_done;

I think parallel scan equivalent bpscan->phs_nblocks along with
hscan->rs_nblocks should be used similar to startblock computation above.

Thank you,
Rahila Syed

On Fri, 29 Mar 2019 at 23:46, Alvaro Herrera <alvherre@2ndquadrant.com>
wrote:

> On 2019-Mar-29, Alvaro Herrera wrote:
>
> > So, CLUSTER and ALTER TABLE rewrites only do non-concurrent index
> > builds; and REINDEX can reuse pretty much the same wait-for metrics
> > columns as CIC.  So I think it's okay if I move only the metrics that
> > conflict for index_build.
>
> The attached version does it that way.  I had to enlarge the param set a
> bit more.  (I suspect those extra columns will be useful to reindex.)
> Also, rebased for recent conflicting changes.
>
>
> I think we should consider a new column of an array type, where we could
> put things like the list of PIDs to be waited for, the list of OIDs of
> index to rebuild, or the list of partitions to build the index on.
>
> --
> Álvaro Herrera                https://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
>


-- 
Rahila Syed
Performance Engineer
2ndQuadrant
http://www.2ndQuadrant.com <http://www.2ndquadrant.com/>
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Commits

  1. Report progress of REINDEX operations

  2. Report progress of CREATE INDEX operations

  3. Fix error handling of readdir() port implementation on first file lookup