Horton

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

Peter Smith <smithpb2250@gmail.com>

View thread
From: Peter Smith <smithpb2250@gmail.com>
To: Tomas Vondra <tomas@vondra.me>
Cc: "Aya Iwata (Fujitsu)" <iwata.aya@fujitsu.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>
Date: 2025-06-10T22:22:46Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Mark ItemPointer arguments as const throughout

  2. Reorganize GUC structs

  3. Eliminate XLOG_HEAP2_VISIBLE from vacuum phase III

  4. Update various forward declarations to use typedef

  5. Pathify RHS unique-ification for semijoin planning

  6. Revert "Don't lock partitions pruned by initial pruning"

  7. For inplace update, send nontransactional invalidations.

  8. Remove nearly-unused SizeOfIptrData macro.

Attachments

On Sat, May 24, 2025 at 7:29 AM Tomas Vondra <tomas@vondra.me> wrote:
>
...
>
> What I really miss in the current patch is some sort of READMEs with
>
> - high-level design of the VCI indexes
>
> - description of the consistency/visibility model (does it behave the
> same way as querying the heap, can it be out of sync for, ...)
>
> - WOS/ROS architecture (when are rows inserted into WOS, promoted into
> ROS, what triggers that, ...)
>
> - what's the in-memory / on-disk format
>
> - some places in the patch mention "internal VCI tables" but I have no
> idea what that is
>
> - how does the execution work? compression? crucial points to consider
> for optimal performance, etc.
>
> - limitations (temporary - can be relaxed in the future, permanent -
> inherent to the columnar design) and trade-offs
>
> - what are the various custom scan executor nodes
>
> - what "background" processes happen (custom workers, ...)
>
> - anything else substantial for understanding the design
>
> Maybe there's some of this in the 0002 patch, but I haven't stumbled
> over it so far.
>

Hi Tomas.

Attached is the first version of the README, intended to address the
points above.

For convenience, I’ve included it as a separate file, but the plan is
to integrate it into the 0002 patch in the next update.

Please let me know if you have any feedback or suggestions for
improving the content.

======
Kind Regards,
Peter Smith.
Fujitsu Australia.