Horton

Re: Parallel CREATE INDEX for BRIN indexes

Tomas Vondra <tomas.vondra@enterprisedb.com>

View thread View on postgr.es
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Matthias van de Meent <boekewurm+postgres@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2023-11-08T11:03:42Z
Lists: pgsql-hackers

Attachments

Hi,

here's an updated patch, addressing the review comments, and reworking
how the work is divided between the workers & leader etc.

0001 is just v2, rebased to current master

0002 and 0003 address most of the issues, in particular it

  - removes the unnecessary spool
  - fixes bs_reltuples type to double
  - a couple comments are reworded to be clearer
  - changes loop/condition in brinbuildCallbackParallel
  - removes asserts added for debugging
  - fixes cast in comparetup_index_brin
  - 0003 then simplifies comparetup_index_brin
  - I haven't inlined the tuplesort_puttuple_common parameter
    (didn't seem worth it)

0004 Reworks how the work is divided between workers and combined by the
leader. It undoes the tableam.c changes that attempted to divide the
relation into chunks matching the BRIN ranges, and instead merges the
results in the leader (using the BRIN "union" function).

I haven't done any indentation fixes yet.

I did fairly extensive testing, using pageinspect to compare indexes
built with/without parallelism. More testing is needed, but it seems to
work fine (with other opclasses and so on).

In general I'm quite happy with the current state, and I believe it's
fairly close to be committable.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. Remove incidental md5() function use from test

  2. Cleanup parallel BRIN index build code

  3. Stabilize test of BRIN parallel create

  4. Revert "Stabilize test of BRIN parallel create"

  5. Add regression test for BRIN parallel builds

  6. Use the correct PG_DETOAST_DATUM macro in BRIN

  7. Update nbits_set in brin_bloom_union

  8. Fix parallel BRIN builds with synchronized scans

  9. Allow parallel CREATE INDEX for BRIN indexes

  10. Add empty BRIN ranges during CREATE INDEX