Horton

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

Peter Smith <smithpb2250@gmail.com>

View thread
From: Peter Smith <smithpb2250@gmail.com>
To: Japin Li <japinli@hotmail.com>
Cc: "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, Tomas Vondra <tomas@vondra.me>, "Aya Iwata (Fujitsu)" <iwata.aya@fujitsu.com>, Timur Magomedov <t.magomedov@postgrespro.ru>, shveta malik <shveta.malik@gmail.com>
Date: 2025-08-01T08:31:18Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Mark ItemPointer arguments as const throughout

  2. Reorganize GUC structs

  3. Eliminate XLOG_HEAP2_VISIBLE from vacuum phase III

  4. Update various forward declarations to use typedef

  5. Pathify RHS unique-ification for semijoin planning

  6. Revert "Don't lock partitions pruned by initial pruning"

  7. For inplace update, send nontransactional invalidations.

  8. Remove nearly-unused SizeOfIptrData macro.

On Wed, Jul 30, 2025 at 4:39 PM Japin Li <japinli@hotmail.com> wrote:
>
> On Wed, 30 Jul 2025 at 14:49, Peter Smith <smithpb2250@gmail.com> wrote:
> > On Fri, Jul 25, 2025 at 1:58 PM Japin Li <japinli@hotmail.com> wrote:
> >>
> >> On Wed, 23 Jul 2025 at 14:07, Peter Smith <smithpb2250@gmail.com> wrote:
> >> > On Tue, Jul 22, 2025 at 8:12 PM Japin Li <japinli@hotmail.com> wrote:
> >> > ...
> > ...
> >> >>
> >> >> Or is it by design that users are unable to read the internal relations?
> >> >>
> >> >
> >> > IIUC, those VCI internal relations (implemented as materialized views)
> >> > are entirely managed by VCI logic. Users are not required to be aware
> >> > of them, and they definitely are not meant to tamper with them.
> >> >
> >>
> >> Thanks for your explanation!
> >>
> >> > The REFRESH that you attempted should have caused a more graceful error, like:
> >> > ERROR:  extension "vci" prohibits this operation on view
> >> > "vci_0000016482_00000_d"
> >> > So, thanks for reporting that the ERROR failed. Investigating...
> >>
> >> I'm considering storing this metadata in heap tables, as Citus Columnar [1]
> >> and TimescaleDB [2] also utilize them for some metadata. Is this a sound
> >> approach? I'm wondering if this is a suitable strategy for VCI?
> >>
> >> [1] https://github.com/citusdata/citus/blob/main/src/backend/columnar/columnar_metadata.c#L174
> >> [2] https://github.com/timescale/timescaledb/blob/main/src/chunk.c#L151
> >>
> >
> > Hi Japin,
> >
> > TL;DR;
> > ------
> > What metadata relations did you have in mind to change, how do you
> > want to change them, and what is the main motivation?
> >
>
> I've noticed a strange user experience with vci_xxx_{d,m}. Although they are
> defined as materialized views, they cannot be queried, leading to an error
> that suggests using REFRESH MATERIALIZED VIEW. However, that command is also
> rejected.  This is my motivation.

FYI, the query is made possible now in v16. The result will be
meaningless, but the error and misleading HINT will no longer happen.

======
Kind Regards,
Peter Smith.
Fujitsu Australia