Horton
Menu

Re: Column Filtering in Logical Replication

Alvaro Herrera <alvherre@alvh.no-ip.org>

View thread
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: Rahila Syed <rahilasyed90@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2021-12-16T20:10:58Z
Lists: pgsql-hackers 
On 2021-Dec-14, Tomas Vondra wrote:

> 7) There's a couple places doing this
> 
>     if (att_map != NULL &&
>         !bms_is_member(att->attnum - FirstLowInvalidHeapAttributeNumber,
>                        att_map) &&
>         !bms_is_member(att->attnum - FirstLowInvalidHeapAttributeNumber,
>                        idattrs) &&
>         !replidentfull)
> 
> which is really hard to understand (even if we get rid of the offset), so
> maybe let's move that to a function with sensible name. Also, some places
> don't check indattrs - seems a bit suspicious.

It is indeed pretty hard to read ... but I think this is completely
unnecessary.  Any column that is part of the identity should have been
included in the column filter, so there is no need to check for the
identity attributes separately.  Testing just for the columns in the
filter ought to be sufficient; and the cases "if att_map NULL" and "is
replica identity FULL" are also equivalent, because in the case of FULL,
you're disallowed from setting a column list.  So this whole thing can
be reduced to just this:

if (att_map != NULL && !bms_is_member(att->attnum, att_map))
       continue;	/* that is, don't send this attribute */

so I don't think this merits a separate function.

[ says he, after already trying to write said function ]

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"Before you were born your parents weren't as boring as they are now. They
got that way paying your bills, cleaning up your room and listening to you
tell them how idealistic you are."  -- Charles J. Sykes' advice to teenagers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Explain about Column List feature.

  2. Doc: fix column list vs. replica identity rules.

  3. Prohibit combining publications with different column lists.

  4. Fix the check to limit sync workers.

  5. Wait for subscription to sync in t/031_column_list.sql

  6. Move prattrs to the pg_publication_rel section in docs

  7. Allow specifying column lists for logical replication

  8. Fix row filters with multiple publications

  9. Fix publish_as_relid with multiple publications

  10. Add some additional tests for row filters in logical replication.

  11. Add index on pg_publication_rel.prpubid

  12. Avoid using DefElemAction in AlterPublicationStmt

  13. Small cleanups related to PUBLICATION framework code

  14. Add PublicationTable and PublicationRelInfo structs

  15. Fix various concurrency issues in logical replication worker launching