Re: Pgoutput not capturing the generated columns

Peter Smith <smithpb2250@gmail.com>

From: Peter Smith <smithpb2250@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Michael Paquier <michael@paquier.xyz>, "Shinoda, Noriyoshi (SXD Japan FSIP)" <noriyoshi.shinoda@hpe.com>, vignesh C <vignesh21@gmail.com>, Shubham Khanna <khannashubham1197@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, Rajendra Kumar Dangwal <dangwalrajendra888@gmail.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, "euler@eulerto.com" <euler@eulerto.com>
Date: 2025-01-15T06:29:41Z
Lists: pgsql-hackers
On Mon, Jan 13, 2025 at 8:27 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Mon, Jan 13, 2025 at 5:25 AM Peter Smith <smithpb2250@gmail.com> wrote:
> >
> > Future -- there probably need to be further clarifications/emphasis to
> > describe how the generated column replication feature only works for
> > STORED generated columns (not VIRTUAL ones), but IMO it is better to
> > address that separately *after* dealing with these missing
> > documentation patches.
> >
>
> I thought it was better to deal with the ambiguity related to the
> 'virtual' part first. I have summarized the options we have regarding
> this in an email [1]. I prefer to extend the current option to take
> values as 's', and 'n'. This will keep the room open to extending it
> with a new value 'v'. The primary reason to go this way is to avoid
> adding new options in the future. It is better to keep the number of
> subscription options under control. Do you have any preference?
>

Hi,

During my review of Vignesh's patch for the enum-version of
publish_generated_columns, I was thinking of yet another way to
specify which columns to replicate.

My idea below is analogous to the existing 'publish' option; Instead
of adding an option specific to generated column types why don't we
instead add a (string) option for controlling the publication of *all*
column types?

Synopsis:

publish_column_types = <col_types>[,...]
where <col_types> are 'normal', 'generated_stored', 'generated_virtual'.

The default value is 'normal', which just means everything that's not generated

~

This option would be overriding if a publication column list is
specified, same as the current implementation does.

~

And, just like the 'publish' option the effect is cumulative:

e.g.1. WITH (publish_column_types = 'normal') == default behavior.
publishes all normal columns same as PG17
e.g.2. WITH (publish_column_types = 'normal, generated_stored') ==
publishes all normal cols AND stored gencols
e.g.3. WITH (publish_column_types = 'generated_stored') == publishes
only the stored gencols and nothing else

Notice that some combinations (like example 3 above with a FOR ALL
TABLES) are not even possible using master, or Vignesh's patch. Maybe
having this extra flexibility is useful for someone?

~

Also, having a generic name 'publish_column_types' leaves this open to
be extended with more possible values in the future without
proliferating more publication options.

Thoughts?

======
Kind Regards,
Peter Smith.
Fujitsu Australia



Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Doc: Generated column replication.

  2. Rename pubgencols_type to pubgencols in pg_publication.

  3. Doc: Fix column name in pg_publication catalog.

  4. Fix buildfarm failure introduced by commit e65dbc9927.

  5. Change publication's publish_generated_columns option type to enum.

  6. Fix \dRp+ output when describing publications with a lower server version.

  7. Replicate generated columns when 'publish_generated_columns' is set.

  8. Doc: Update the behavior of generated columns in Logical Replication.

  9. Replicate generated columns when specified in the column list.

  10. Doc: Generated columns are skipped for logical replication.