Horton

Thread

Commits

  1. Update comments about index parallel builds

  2. doc: Mention BRIN indexes support parallel builds

  3. Allow parallel CREATE INDEX for BRIN indexes

  1. Parallel index build for BRIN

    Egor Rogov <e.rogov@postgrespro.ru> — 2024-11-05T09:12:14Z

    Hi,

Commit b4375717 introduced parallel CREATE INDEX for BRIN. I've noticed 
that a couple of documentation pages need to be updated accordingly. A 
small patch is attached.

Thanks,
Egor

  2. Re: Parallel index build for BRIN

    Egor Rogov <e.rogov@postgrespro.ru> — 2024-11-17T08:28:07Z

    Hi everyone,

Thisthread doesn'tseem to have attractedattention, so let me try again. 
Two documentation pages claim that B-tree is the only access method that 
supports parallel building, which is no longer true. I propose to fix it 
in a way like this:

diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index d54f9049569..b5b1580dee7 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2835,7 +2835,7 @@ include_dir 'conf.d'
           Sets the maximum number of parallel workers that can be
           started by a single utility command.  Currently, the parallel
           utility commands that support the use of parallel workers are
-         <command>CREATE INDEX</command> only when building a B-tree index,
+         <command>CREATE INDEX</command> when building a B-tree or BRIN 
index,
           and <command>VACUUM</command> without <literal>FULL</literal>
           option.  Parallel workers are taken from the pool of processes
           established by <xref linkend="guc-max-worker-processes"/>, 
limited
diff --git a/doc/src/sgml/ref/create_index.sgml 
b/doc/src/sgml/ref/create_index.sgml
index 621bc0e253c..208389e8006 100644
--- a/doc/src/sgml/ref/create_index.sgml
+++ b/doc/src/sgml/ref/create_index.sgml
@@ -808,7 +808,7 @@ Indexes:
     leveraging multiple CPUs in order to process the table rows faster.
     This feature is known as <firstterm>parallel index
     build</firstterm>.  For index methods that support building indexes
-   in parallel (currently, only B-tree),
+   in parallel (currently, B-tree and BRIN),
     <varname>maintenance_work_mem</varname> specifies the maximum
     amount of memory that can be used by each index build operation as
     a whole, regardless of how many worker processes were started.

Thanks,
Egor


On 05.11.2024 12:12, Egor Rogov wrote:
> Hi,
>
> Commit b4375717 introduced parallel CREATE INDEX for BRIN. I've 
> noticed that a couple of documentation pages need to be updated 
> accordingly. A small patch is attached.
>
> Thanks,
> Egor

  3. Re: Parallel index build for BRIN

    Egor Rogov <e.rogov@postgrespro.ru> — 2024-12-08T15:00:25Z

    Hi,


On 17.11.2024 11:28, Egor Rogov wrote:
> Hi everyone,
>
> This thread doesn't seem to have attracted attention, so let me try 
> again. Two documentation pages claim that B-tree is the only access 
> method that supports parallel building, which is no longer true. I 
> propose to fix it in a way like this:
>
> diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
> index d54f9049569..b5b1580dee7 100644
> --- a/doc/src/sgml/config.sgml
> +++ b/doc/src/sgml/config.sgml
> @@ -2835,7 +2835,7 @@ include_dir 'conf.d'
>           Sets the maximum number of parallel workers that can be
>           started by a single utility command.  Currently, the parallel
>           utility commands that support the use of parallel workers are
> -         <command>CREATE INDEX</command> only when building a B-tree 
> index,
> +         <command>CREATE INDEX</command> when building a B-tree or 
> BRIN index,
>           and <command>VACUUM</command> without <literal>FULL</literal>
>           option.  Parallel workers are taken from the pool of processes
>           established by <xref linkend="guc-max-worker-processes"/>, 
> limited
> diff --git a/doc/src/sgml/ref/create_index.sgml 
> b/doc/src/sgml/ref/create_index.sgml
> index 621bc0e253c..208389e8006 100644
> --- a/doc/src/sgml/ref/create_index.sgml
> +++ b/doc/src/sgml/ref/create_index.sgml
> @@ -808,7 +808,7 @@ Indexes:
>     leveraging multiple CPUs in order to process the table rows faster.
>     This feature is known as <firstterm>parallel index
>     build</firstterm>.  For index methods that support building indexes
> -   in parallel (currently, only B-tree),
> +   in parallel (currently, B-tree and BRIN),
>     <varname>maintenance_work_mem</varname> specifies the maximum
>     amount of memory that can be used by each index build operation as
>     a whole, regardless of how many worker processes were started.


I've spotted another mention of B-tree being the only AM that supports 
parallel builds: comment in src/backend/catalog/index.c. As this mention 
is not visible to the users, I'd propose removing it altogether rather 
than fixing it. Updated patch is attached.


>
> Thanks,
> Egor
>
>
> On 05.11.2024 12:12, Egor Rogov wrote:
>> Hi,
>>
>> Commit b4375717 introduced parallel CREATE INDEX for BRIN. I've 
>> noticed that a couple of documentation pages need to be updated 
>> accordingly. A small patch is attached.
>>
>> Thanks,
>> Egor

  4. Re: Parallel index build for BRIN

    Tomas Vondra <tomas@vondra.me> — 2024-12-09T18:54:42Z

    On 12/8/24 16:00, Egor Rogov wrote:
> Hi,
> 
> 
> On 17.11.2024 11:28, Egor Rogov wrote:
>> Hi everyone,
>>
>> This thread doesn't seem to have attracted attention, so let me try
>> again. Two documentation pages claim that B-tree is the only access
>> method that supports parallel building, which is no longer true. I
>> propose to fix it in a way like this:
>>
>> diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
>> index d54f9049569..b5b1580dee7 100644
>> --- a/doc/src/sgml/config.sgml
>> +++ b/doc/src/sgml/config.sgml
>> @@ -2835,7 +2835,7 @@ include_dir 'conf.d'
>>           Sets the maximum number of parallel workers that can be
>>           started by a single utility command.  Currently, the parallel
>>           utility commands that support the use of parallel workers are
>> -         <command>CREATE INDEX</command> only when building a B-tree
>> index,
>> +         <command>CREATE INDEX</command> when building a B-tree or
>> BRIN index,
>>           and <command>VACUUM</command> without <literal>FULL</literal>
>>           option.  Parallel workers are taken from the pool of processes
>>           established by <xref linkend="guc-max-worker-processes"/>,
>> limited
>> diff --git a/doc/src/sgml/ref/create_index.sgml b/doc/src/sgml/ref/
>> create_index.sgml
>> index 621bc0e253c..208389e8006 100644
>> --- a/doc/src/sgml/ref/create_index.sgml
>> +++ b/doc/src/sgml/ref/create_index.sgml
>> @@ -808,7 +808,7 @@ Indexes:
>>     leveraging multiple CPUs in order to process the table rows faster.
>>     This feature is known as <firstterm>parallel index
>>     build</firstterm>.  For index methods that support building indexes
>> -   in parallel (currently, only B-tree),
>> +   in parallel (currently, B-tree and BRIN),
>>     <varname>maintenance_work_mem</varname> specifies the maximum
>>     amount of memory that can be used by each index build operation as
>>     a whole, regardless of how many worker processes were started.
> 
> 
> I've spotted another mention of B-tree being the only AM that supports
> parallel builds: comment in src/backend/catalog/index.c. As this mention
> is not visible to the users, I'd propose removing it altogether rather
> than fixing it. Updated patch is attached.
> 

Thanks for noticing this and the patches. You're right, this should have
been updated with the BRIN parallel builds. I'll get this committed
sometime the week.


regards

-- 
Tomas Vondra

  5. Re: Parallel index build for BRIN

    Tomas Vondra <tomas@vondra.me> — 2024-12-16T18:24:42Z

    On 12/9/24 19:54, Tomas Vondra wrote:
> On 12/8/24 16:00, Egor Rogov wrote:
>> Hi,
>>
>> ...
>>
>> I've spotted another mention of B-tree being the only AM that supports
>> parallel builds: comment in src/backend/catalog/index.c. As this mention
>> is not visible to the users, I'd propose removing it altogether rather
>> than fixing it. Updated patch is attached.
>>
> 
> Thanks for noticing this and the patches. You're right, this should have
> been updated with the BRIN parallel builds. I'll get this committed
> sometime the week.
> 

I've pushed the doc fix, and backpatched it to PG 17.

Thanks for the patience!

-- 
Tomas Vondra

  6. Re: Parallel index build for BRIN

    Egor Rogov <e.rogov@postgrespro.ru> — 2024-12-17T06:40:31Z

    On 16.12.2024 21:24, Tomas Vondra wrote:

> On 12/9/24 19:54, Tomas Vondra wrote:
>> On 12/8/24 16:00, Egor Rogov wrote:
>>> Hi,
>>>
>>> ...
>>>
>>> I've spotted another mention of B-tree being the only AM that supports
>>> parallel builds: comment in src/backend/catalog/index.c. As this mention
>>> is not visible to the users, I'd propose removing it altogether rather
>>> than fixing it. Updated patch is attached.
>>>
>> Thanks for noticing this and the patches. You're right, this should have
>> been updated with the BRIN parallel builds. I'll get this committed
>> sometime the week.
>>
> I've pushed the doc fix, and backpatched it to PG 17.


Thanks so much, Tomas!

Please note that the comment in src/backend/catalog/index.c remains 
unchanged:


--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2988,8 +2988,7 @@ index_build(Relation heapRelation,
Assert(PointerIsValid(indexRelation->rd_indam->ambuildempty));

     /*
-    * Determine worker process details for parallel CREATE INDEX.  
Currently,
-    * only btree has support for parallel builds.
+    * Determine worker process details for parallel CREATE INDEX.
      *
      * Note that planner considers parallel safety for us.
      */

  7. Re: Parallel index build for BRIN

    Tomas Vondra <tomas@vondra.me> — 2024-12-17T14:52:25Z

    On 12/17/24 07:40, Egor Rogov wrote:
> On 16.12.2024 21:24, Tomas Vondra wrote:
> 
>> On 12/9/24 19:54, Tomas Vondra wrote:
>>> On 12/8/24 16:00, Egor Rogov wrote:
>>>> Hi,
>>>>
>>>> ...
>>>>
>>>> I've spotted another mention of B-tree being the only AM that supports
>>>> parallel builds: comment in src/backend/catalog/index.c. As this
>>>> mention
>>>> is not visible to the users, I'd propose removing it altogether rather
>>>> than fixing it. Updated patch is attached.
>>>>
>>> Thanks for noticing this and the patches. You're right, this should have
>>> been updated with the BRIN parallel builds. I'll get this committed
>>> sometime the week.
>>>
>> I've pushed the doc fix, and backpatched it to PG 17.
> 
> 
> Thanks so much, Tomas!
> 
> Please note that the comment in src/backend/catalog/index.c remains
> unchanged:
> 
> 
> --- a/src/backend/catalog/index.c
> +++ b/src/backend/catalog/index.c
> @@ -2988,8 +2988,7 @@ index_build(Relation heapRelation,
> Assert(PointerIsValid(indexRelation->rd_indam->ambuildempty));
> 
>     /*
> -    * Determine worker process details for parallel CREATE INDEX. 
> Currently,
> -    * only btree has support for parallel builds.
> +    * Determine worker process details for parallel CREATE INDEX.
>      *
>      * Note that planner considers parallel safety for us.
>      */
> 

Oh, thanks for noticing that. I'm not sure why I missed that, I must
have used the first patch by mistake.

Anyway, I found another obsolete comment in planner.c, so I fixed that
too and pushed.


Thanks

-- 
Tomas Vondra