Re: [WIP]Vertical Clustered Index (columnar store extension) - take2
Peter Smith <smithpb2250@gmail.com>
From: Peter Smith <smithpb2250@gmail.com>
To: Tomas Vondra <tomas@vondra.me>
Cc: "Aya Iwata (Fujitsu)" <iwata.aya@fujitsu.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>
Date: 2025-05-29T08:08:20Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Mark ItemPointer arguments as const throughout
- e1ac846f3d28 19 (unreleased) cited
-
Reorganize GUC structs
- a13833c35f9e 19 (unreleased) cited
-
Eliminate XLOG_HEAP2_VISIBLE from vacuum phase III
- add323da40a6 19 (unreleased) cited
-
Update various forward declarations to use typedef
- d4d1fc527bdb 19 (unreleased) cited
-
Pathify RHS unique-ification for semijoin planning
- 24225ad9aafc 19 (unreleased) cited
-
Revert "Don't lock partitions pruned by initial pruning"
- 1722d5eb05d8 18.0 cited
-
For inplace update, send nontransactional invalidations.
- 243e9b40f1b2 18.0 cited
-
Remove nearly-unused SizeOfIptrData macro.
- 8023b5827fba 10.0 cited
Attachments
- v4-0003-VCI-documentation.patch (application/octet-stream) patch v4-0003
- v4-0001-Add-postgres-code-fix-needed-for-VCI.patch (application/octet-stream) patch v4-0001
- v4-0002-Add-VCI-module.patch (application/octet-stream) patch v4-0002
Hi Tomas. > What I really miss in the current patch is some sort of READMEs with > > - high-level design of the VCI indexes > > - description of the consistency/visibility model (does it behave the > same way as querying the heap, can it be out of sync for, ...) > > - WOS/ROS architecture (when are rows inserted into WOS, promoted into > ROS, what triggers that, ...) > > - what's the in-memory / on-disk format > > - some places in the patch mention "internal VCI tables" but I have no > idea what that is > > - how does the execution work? compression? crucial points to consider > for optimal performance, etc. > > - limitations (temporary - can be relaxed in the future, permanent - > inherent to the columnar design) and trade-offs > > - what are the various custom scan executor nodes > > - what "background" processes happen (custom workers, ...) > > - anything else substantial for understanding the design > > Maybe there's some of this in the 0002 patch, but I haven't stumbled > over it so far. Acknowledged. We are working now on creating a draft README and will post it here ASAP. ~~~ For patch 0001 you reported some items as being "unused". You were correct on all counts; those are all removed in the latest patch set. > gen_node_support.c > ------------------ > - What's SmcAllocSetContext for? Doesn't seem to be mentioned anywhere. Removed. > params.c > ------------------ > - ExecInitParam_hook seems unused, no? Removed. > lwlock.c > ------------------ > - Vci_lwlock_kind seems unused. Removed. > memnodes.h > ------------------ > - What's SmcAllocSetContext? It's not defined anywhere. Removed. > params.h > ------------------ > - Adds noNeedLoadFromMain, loadedFromMain to ParamExecData, but it > doesn't seem to be used anywhere (in either patch). Removed. > lwlock.h > ------------------ > - LWLockKind / Vci_lwlock_kind seems unused Removed ====== Kind Regards, Peter Smith. Fujitsu Australia.