Horton

Re: Support logical replication of DDLs

Zheng Li <zhengli10@gmail.com>

View thread
From: Zheng Li <zhengli10@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Peter Smith <smithpb2250@gmail.com>, vignesh C <vignesh21@gmail.com>, li jie <ggysxcq@gmail.com>, Ajin Cherian <itsajin@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Masahiko Sawada <sawada.mshk@gmail.com>, Japin Li <japinli@hotmail.com>, rajesh singarapu <rajesh.rs0541@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2023-01-19T03:09:02Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add a run_as_owner option to subscriptions.

  2. Refactor pgoutput_change().

  3. Print the correct aliases for DML target tables in ruleutils.

  4. Fix object identity string for transforms

  5. Add grantable MAINTAIN privilege and pg_maintain role.

  6. Get rid of recursion-marker values in enum AlterTableType

  7. Release cache tuple when no longer needed

  8. Empty search_path in logical replication apply worker and walsender.

  9. Refactor format_type APIs to be more modular

  10. Use wrappers of PG_DETOAST_DATUM_PACKED() more.

On Wed, Jan 18, 2023 at 6:27 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Sat, Jan 7, 2023 at 8:58 PM Zheng Li <zhengli10@gmail.com> wrote:
> >
> > I added documentation and changed user interface design in the
> > attached v60 patch set.
> > The patch set addressed comments from Peter in [1].
> >
> > The motivation for the user interface change is that we want to manage
> > DDL replication feature in stages with fine grained replication
> > levels.
> > For example, we can focus on reviewing and testing table commands
> > first, then other commands. It also make sense to introduce different
> > DDL replication levels
> > from the user perspective as pointed out in [1]. We can add more
> > replication levels along the way.
> >
> > In this patch DDL replication is disabled by default and it can be
> > enabled at different levels
> > using the new PUBLICATION option 'ddl'. This option currently has two
> > levels and are
> > only allowed to be set if the PUBLICATION is FOR ALL TABLES.
> >
> >   all: this option enables replication of all supported DDL commands.
> >   table: this option enables replication of Table DDL commands which include:
> >   -CREATE/ALTER/DROP TABLE
> >   -CREATE TABLE AS
> >
>
> I think this point needs some thought. When you say 'all', how do you
> think it will help to support DDL replication for foreign tables,
> materialized views, views, etc where changes to such relations are
> currently not supported by logical replication?

I think DDL replication naturally provides support for views and
materialized views,
if the publication is FOR ALL TABLES since all the tables in the
view/MV definition
are replicated.

Foreign Tables can also be considered replicated with DDL replication because we
don't even need to replicate the data as it resides on the external
server. Users
need to configure the external server to allow connection from the
subscriber for
foreign tables to work on the subscriber.

> We should also think
> about initial sync for all those objects as well.

Agree, we're starting an investigation on initial sync. But I think
initial sync depends on
DDL replication to work reliably, not the other way around. DDL replication can
work on its own without the initial sync of schema, users just need to
setup the initial
schema just like they would today.

Regards,
Znae