Thread

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

Aya Iwata (Fujitsu) <iwata.aya@fujitsu.com> — 2025-05-17T16:31:30Z

Hi hackers,

I will share the notes on the discussions in PGConf.dev. Thanks all for participation.
 
Feedback on the Advanced Patch Feedback Session;
 
- A Basic idea that allows both OLTP/OLAP workloads on the same table is OK
  - Buffering mechanism has already been used by existing code GIN, IIUC
- IAM seems a more proper approach than TAM
  - One reason is that VCI can only optimize the data lookup stuff
  - TAM needs all possible queries; it's too much
- Divide the patch more and more
  - to allow committers to consider pros and cons and push one by one
  - 15 patches is the maximum amount
  - Separate features to some committable group
 
Sawada san suggested several points;
1. Find parts that are useful not only for the VCI and dispatching new threads
2. Develop codes incrementally. E.g., VCI has a custom scan, but we may
be able to give up on the first version
 

Also, I have questions and advice below on May 16th.
 
- How to handle visibility?
  - What if transactions that insert tuples are aborted?
- VCI vs Index Only Scan
  - VCI seems the same as Index Only Scan
  - VCI has compression. So, is the amount of size better than the Index?
- How about the efficiency for storage? How much data can we store?
- using TAM (Alvalo's suggestion)
  - Most of the code can be ported from the heap. ROS and WOS can exist as forks
    of main files, like .vm and .fm. API can be like:
 
```
CREATE TABLE foo (id, product, price) USING vci WITH (to_be_indexed=price);
```

Again, I really appreciate your efforts. 


Regards,
Aya Iwata
Fujitsu Limited