Re: [WIP]Vertical Clustered Index (columnar store extension) - take2
shveta malik <shveta.malik@gmail.com>
From: shveta malik <shveta.malik@gmail.com>
To: Peter Smith <smithpb2250@gmail.com>
Cc: Timur Magomedov <t.magomedov@postgrespro.ru>,
Tomas Vondra <tomas@vondra.me>, "Aya Iwata (Fujitsu)" <iwata.aya@fujitsu.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>,
Japin Li <japinli@hotmail.com>, shveta malik <shveta.malik@gmail.com>
Date: 2025-07-11T03:45:54Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Mark ItemPointer arguments as const throughout
- e1ac846f3d28 19 (unreleased) cited
-
Reorganize GUC structs
- a13833c35f9e 19 (unreleased) cited
-
Eliminate XLOG_HEAP2_VISIBLE from vacuum phase III
- add323da40a6 19 (unreleased) cited
-
Update various forward declarations to use typedef
- d4d1fc527bdb 19 (unreleased) cited
-
Pathify RHS unique-ification for semijoin planning
- 24225ad9aafc 19 (unreleased) cited
-
Revert "Don't lock partitions pruned by initial pruning"
- 1722d5eb05d8 18.0 cited
-
For inplace update, send nontransactional invalidations.
- 243e9b40f1b2 18.0 cited
-
Remove nearly-unused SizeOfIptrData macro.
- 8023b5827fba 10.0 cited
Thank You for working on this. I started going through the README and
tried running simple tests, have few concerns:
1)
I am not able to understand section 4.2 'WOS-to-ROS conversion'. When
whiteout-WOS says 'delete 4', what does that mean? 4 is CRID, TXID?
And when does delete-vector X represents? I did not get why ColA-2,
ColA-4 and ColB-2, ColB-4 were removed in resultant data? Is the
diagram complete?
2)
We can make the definition consistent at both places as the first one
gives a feeling that rows are marked for deletion in WOS while the
second one says ROS.
Whiteout WOS = Record of WOS rows marked for deletion
Whiteout WOS -- TID records of WOS rows that are marked for deletion on ROS
3)
It is not part of README. But please help me understand the meaning
and usage of this GUC in VCI context:
vci.max_devices: Sets the maximum device number which can be attached.
4)
I was trying my hands on using VCI, I found that on enabling the VCI
path, autovacuum asserts in between. We can reproduce the scenario
using:
create extension vci;
create table tab1( i int, j int, k int);
create index ivci on tab1 using vci (i);
insert into tab1 SELECT generate_series(1,1000),
generate_series(1,1000), generate_series(1,1000);
insert into tab1 SELECT generate_series(1,1000),
generate_series(1,1000), generate_series(1,1000);
SET enable_seqscan TO off;
VACUUM ANALYZE public.tab1;
It asserts here:
TRAP: failed Assert("TransactionIdIsNormal(xmax)"), File: "heapam.c",
Line: 7373, PID: 79834
postgres: autovacuum worker postgres(ExceptionalCondition+0xbb)[0x55fdf8f3ccc6]
postgres: autovacuum worker
postgres(heap_pre_freeze_checks+0x18f)[0x55fdf877e26f]
postgres: autovacuum worker postgres(index_vacuum_cleanup+0x181)[0x55fdf87a1c06]
postgres: autovacuum worker postgres(vac_cleanup_one_index+0x27)[0x55fdf8a0503a]
-------------
Few typos in README:
a) Each VCI indexed column is stored as an internal relations.
--relations --> relation
b) Records are addresses by CRID (Columnar Record ID) instead of by TID.
--addresses->addressed
c) Extents can be found by ID using offsets in a column "meta-data"
internal relation.
-- by ID using offsets? Do you mean 'by using offsets' alone?
d) EXPLAIN ANALYSE -->EXPLAIN ANALYZE
thanks
Shveta