Horton

Re: Support logical replication of DDLs

Zheng Li <zhengli10@gmail.com>

View thread
From: Zheng Li <zhengli10@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>, Euler Taveira <euler@eulerto.com>, japin <japinli@hotmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-04-13T17:22:46Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add a run_as_owner option to subscriptions.

  2. Refactor pgoutput_change().

  3. Print the correct aliases for DML target tables in ruleutils.

  4. Fix object identity string for transforms

  5. Add grantable MAINTAIN privilege and pg_maintain role.

  6. Get rid of recursion-marker values in enum AlterTableType

  7. Release cache tuple when no longer needed

  8. Empty search_path in logical replication apply worker and walsender.

  9. Refactor format_type APIs to be more modular

  10. Use wrappers of PG_DETOAST_DATUM_PACKED() more.

Attachments

>Thanks, but it would be good if you can share it in the patch form as
>well unless you need to send patches too frequently. It is easier to
>give comments.

The rebased patches are attached. They apply on commit ed0fbc8.

> > I agree with the point that their design can not be completely
> > independent.  They have some logical relationship of what schema will
> > be copied by the initial sync and where is the exact boundary from
> > which we will start sending as replication.  And suppose first we only
> > plan to implement the replication part then how the user will know
> > what all schema user has to create and what will be replicated using
> > DDL replication?  Suppose the user takes a dump and copies all the
> > schema and then creates the subscription, then how we are we going to
> > handle the DDL concurrent to the subscription command?
> >
>
> Right, I also don't see how it can be done in the current
> implementation. So, I think even if we want to develop these two as
> separate patches they need to be integrated to make the solution
> complete.

The concurrent DDL issue actually exists today even with DML
replication. i.e. user have to make sure table schema is in sync
before starting logical replication.

Here is a high level idea to implement schema initialization:
1. On source DB: dump the schema and create a logical replication slot
on a snapshot at the same time
2. On target DB: restore the schema
3. On target DB: create subscription by pointing it to the logical
replication slot and the snapshot in step 1, so that the initial sync
is based on the snapshot and concurrent changes are replicated using
the logical replication slot.

Regards,
Zheng