Horton

Re: Support logical replication of DDLs

Amit Kapila <amit.kapila16@gmail.com>

View thread
From: Amit Kapila <amit.kapila16@gmail.com>
To: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
Cc: shveta malik <shveta.malik@gmail.com>, vignesh C <vignesh21@gmail.com>, Ajin Cherian <itsajin@gmail.com>, "Wei Wang (Fujitsu)" <wangw.fnst@fujitsu.com>, Runqi Tian <runqidev@gmail.com>, Peter Smith <smithpb2250@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, li jie <ggysxcq@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Masahiko Sawada <sawada.mshk@gmail.com>, Japin Li <japinli@hotmail.com>, rajesh singarapu <rajesh.rs0541@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Zheng Li <zhengli10@gmail.com>
Date: 2023-04-28T11:41:03Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add a run_as_owner option to subscriptions.

  2. Refactor pgoutput_change().

  3. Print the correct aliases for DML target tables in ruleutils.

  4. Fix object identity string for transforms

  5. Add grantable MAINTAIN privilege and pg_maintain role.

  6. Get rid of recursion-marker values in enum AlterTableType

  7. Release cache tuple when no longer needed

  8. Empty search_path in logical replication apply worker and walsender.

  9. Refactor format_type APIs to be more modular

  10. Use wrappers of PG_DETOAST_DATUM_PACKED() more.

On Tue, Apr 25, 2023 at 9:28 AM Zhijie Hou (Fujitsu)
<houzj.fnst@fujitsu.com> wrote:
>

I have a few high-level comments on the deparsing approach used in the
patch. As per my understanding, we first build an ObjTree from the DDL
command, then convert the ObjTree to Jsonb which is then converted to
a JSON string.  Now, in the consecutive patch, via publication event
triggers, we get the JSON string via the conversions mentioned, WAL
log it, which then walsender will send to the subscriber, which will
convert the JSON string back to the DDL command and execute it.

Now, I think we can try to eliminate this entire ObjTree machinery and
directly from the JSON blob during deparsing. We have previously also
discussed this in an email chain at [1]. I think now the functionality
of JSONB has also been improved and we should investigate whether it
is feasible to directly use JSONB APIs to form the required blob.

The other general point is that one of the primary reasons to convert
DDL into JSONB blob is to allow replacing the elements. For example,
say on the publisher, the table is in Schema A and then on the
subscriber the same table is in Schema B, so, we would like to change
the Schema in the DDL before replaying it, or we want to change the
persistence of table to UNLOGGED before replaying the DDL on the
subscriber. Is it possible to have such an API exposed from this
module so that we can verify if that works? It can be a separate patch
though.

[1] - https://www.postgresql.org/message-id/CAB7nPqRX6w9UY%2B%3DOy2jqTVwi0hqT2y4%3DfUc7fNG4U-296JBvYQ%40mail.gmail.com

-- 
With Regards,
Amit Kapila.