Horton

Thread

  1. [Patch] Omit virtual generated columns from test_decoding output

    SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com> — 2026-05-05T01:11:11Z

    Hi Hackers,

Virtual generated columns are not stored on disk, so heap_getattr() in
tuple_to_stringinfo() always returned NULL for them, producing
misleading output such as

  table public.t: INSERT: a[integer]:1 b[integer]:10 c[integer]:null

even though the user could observe a non-null value via SELECT.  Stored
generated columns continue to be emitted as before because their values
do live in the heap tuple.

This matches the pgoutput's logicalrep_should_publish_column()
which never publishes virtual generated columns. Added a regression test.
Please find the patch attached.

Thanks,
Satya

  2. Re: [Patch] Omit virtual generated columns from test_decoding output

    Euler Taveira <euler@eulerto.com> — 2026-05-05T03:08:58Z

    On Mon, May 4, 2026, at 10:11 PM, SATYANARAYANA NARLAPURAM wrote:
>
> Virtual generated columns are not stored on disk, so heap_getattr() in
> tuple_to_stringinfo() always returned NULL for them, producing
> misleading output such as
>
>   table public.t: INSERT: a[integer]:1 b[integer]:10 c[integer]:null
>
> even though the user could observe a non-null value via SELECT.  Stored
> generated columns continue to be emitted as before because their values
> do live in the heap tuple.
>

I wouldn't say misleading but expected. Logical decoding relies on WAL and
virtual generated columns are not stored in the WAL.

> This matches the pgoutput's logicalrep_should_publish_column()
> which never publishes virtual generated columns. Added a regression test.
> Please find the patch attached.
>

There is no guarantee that test_decoding should match the pgoutput. I agree that
test_decoding shouldn't output virtual generated columns. The problem is that it
already does it. I'm afraid that removing it should break existing applications.
(I heard that some solutions rely on test_decoding for CDC.) Should we change it
as you proposed or add an option to put it back to keep the old behavior?

I didn't review your patch but I noticed that there is a new test file for this
change. There are some concerns about the total test execution time. Do you
really need to include this test? If so, should you combine it with an existing
test file?


-- 
Euler Taveira
EDB   https://www.enterprisedb.com/

  3. Re: [Patch] Omit virtual generated columns from test_decoding output

    SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com> — 2026-05-05T05:16:26Z

    Hi,

On Mon, May 4, 2026 at 8:09 PM Euler Taveira <euler@eulerto.com> wrote:

> On Mon, May 4, 2026, at 10:11 PM, SATYANARAYANA NARLAPURAM wrote:
> >
> > Virtual generated columns are not stored on disk, so heap_getattr() in
> > tuple_to_stringinfo() always returned NULL for them, producing
> > misleading output such as
> >
> >   table public.t: INSERT: a[integer]:1 b[integer]:10 c[integer]:null
> >
> > even though the user could observe a non-null value via SELECT.  Stored
> > generated columns continue to be emitted as before because their values
> > do live in the heap tuple.
> >
>
> I wouldn't say misleading but expected. Logical decoding relies on WAL and
> virtual generated columns are not stored in the WAL.
>
> > This matches the pgoutput's logicalrep_should_publish_column()
> > which never publishes virtual generated columns. Added a regression test.
> > Please find the patch attached.
> >
>
> There is no guarantee that test_decoding should match the pgoutput.


Agreed, not trying to keep them in sync but giving as a reference.



> I agree that
> test_decoding shouldn't output virtual generated columns. The problem is
> that it
> already does it. I'm afraid that removing it should break existing
> applications.
> (I heard that some solutions rely on test_decoding for CDC.) Should we
> change it
> as you proposed or add an option to put it back to keep the old behavior?
>

It is emitting null, I am not sure if it is meaningful for the consumers to
consume this or
have taken dependency on this. Adding an extra option isn't an overkill for
this? I am open
to this idea if others feel the same.



> I didn't review your patch but I noticed that there is a new test file for
> this
> change. There are some concerns about the total test execution time. Do you
> really need to include this test? If so, should you combine it with an
> existing
> test file?


Fair concern, I moved the tests to ddl.sql.  Please find the attached v2
patch.

Thanks,
Satya

  4. Re: [Patch] Omit virtual generated columns from test_decoding output

    Fujii Masao <masao.fujii@gmail.com> — 2026-05-08T04:31:01Z

    On Tue, May 5, 2026 at 2:16 PM SATYANARAYANA NARLAPURAM
<satyanarlapuram@gmail.com> wrote:
> Fair concern, I moved the tests to ddl.sql.  Please find the attached v2 patch.

Seems you forgot to attached the patch.

Regards,

-- 
Fujii Masao

  5. Re: [Patch] Omit virtual generated columns from test_decoding output

    SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com> — 2026-05-09T20:58:38Z

    Hi,

On Thu, May 7, 2026 at 9:31 PM Fujii Masao <masao.fujii@gmail.com> wrote:

> On Tue, May 5, 2026 at 2:16 PM SATYANARAYANA NARLAPURAM
> <satyanarlapuram@gmail.com> wrote:
> > Fair concern, I moved the tests to ddl.sql.  Please find the attached v2
> patch.
>
> Seems you forgot to attached the patch.
>

Attached now, thanks for letting ne know!

  6. Re: [Patch] Omit virtual generated columns from test_decoding output

    SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com> — 2026-05-09T21:00:20Z

    Hi

On Sat, May 9, 2026 at 1:58 PM SATYANARAYANA NARLAPURAM <
satyanarlapuram@gmail.com> wrote:

> Hi,
>
> On Thu, May 7, 2026 at 9:31 PM Fujii Masao <masao.fujii@gmail.com> wrote:
>
>> On Tue, May 5, 2026 at 2:16 PM SATYANARAYANA NARLAPURAM
>> <satyanarlapuram@gmail.com> wrote:
>> > Fair concern, I moved the tests to ddl.sql.  Please find the attached
>> v2 patch.
>>
>> Seems you forgot to attached the patch.
>>
>
> Attached now, thanks for letting ne know!
>

Please find the v3 patch.

Thanks,
Satya