Re: Parallel CREATE INDEX for GIN indexes

Tomas Vondra <tomas.vondra@enterprisedb.com>

View thread

From: Tomas Vondra <tomas.vondra@enterprisedb.com>

To: pgsql-hackers@lists.postgresql.org

Cc: Andy Fan <zhihuifan1213@163.com>

Date: 2024-06-20T21:19:43Z

Lists: pgsql-hackers

Attachments

v20240620-0001-Allow-parallel-create-for-GIN-indexes.patch (text/x-patch) patch v20240620-0001
v20240620-0002-Use-mergesort-in-the-leader-process.patch (text/x-patch) patch v20240620-0002
v20240620-0003-Remove-the-explicit-pg_qsort-in-workers.patch (text/x-patch) patch v20240620-0003
v20240620-0004-Compress-TID-lists-before-writing-tuples-t.patch (text/x-patch) patch v20240620-0004
v20240620-0005-Collect-and-print-compression-stats.patch (text/x-patch) patch v20240620-0005
v20240620-0006-Enforce-memory-limit-when-combining-tuples.patch (text/x-patch) patch v20240620-0006
v20240620-0007-Detect-wrap-around-in-parallel-callback.patch (text/x-patch) patch v20240620-0007

Here's a cleaned up patch series, merging the fixup patches into 0001.

I've also removed the memset() from ginInsertBAEntry(). This was meant
to fix valgrind reports, but I believe this was just a symptom of
incorrect handling of byref data types, which was fixed in 2024/05/02
patch version.

The other thing I did is cleanup of FIXME and XXX comments. There were a
couple stale/obsolete comments, discussing issues that have been already
fixed (like the scan wrapping around).

A couple things to fix remain, but all of them are minor. And there's
also a couple XXX comments, often describing thing that is then done in
one of the following patches.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Exercise parallel GIN builds in regression tests
- f6e5d21bf73f 19 (unreleased) landed
- 32593394ee43 18.2 landed
Lookup the correct ordering for parallel GIN builds
- db14dcdec69d 19 (unreleased) landed
- eee71a66cc86 18.2 landed
Fix memory leak in _gin_parallel_merge
- 1681a70df3d6 18.0 landed
Enforce memory limit during parallel GIN builds
- b229c1016477 18.0 landed
Compress TID lists when writing GIN tuples to disk
- 0b2a45a5d1f2 18.0 landed
Allow parallel CREATE INDEX for GIN indexes
- 8492feb98f6d 18.0 landed