Horton

Re: Support logical replication of DDLs

Konstantin Berkaev <konstantinberkaev@gmail.com>

View thread
From: Konstantin Berkaev <konstantinberkaev@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: "Andrey M. Borodin" <x4mmm@yandex-team.ru>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, Masahiko Sawada <sawada.mshk@gmail.com>, shveta malik <shveta.malik@gmail.com>, Michael Paquier <michael@paquier.xyz>, "Wei Wang (Fujitsu)" <wangw.fnst@fujitsu.com>, "Yu Shi (Fujitsu)" <shiy.fnst@fujitsu.com>, vignesh C <vignesh21@gmail.com>, Ajin Cherian <itsajin@gmail.com>, Runqi Tian <runqidev@gmail.com>, Peter Smith <smithpb2250@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, li jie <ggysxcq@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Japin Li <japinli@hotmail.com>, rajesh singarapu <rajesh.rs0541@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Zheng Li <zhengli10@gmail.com>
Date: 2024-07-25T05:42:34Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add a run_as_owner option to subscriptions.

  2. Refactor pgoutput_change().

  3. Print the correct aliases for DML target tables in ruleutils.

  4. Fix object identity string for transforms

  5. Add grantable MAINTAIN privilege and pg_maintain role.

  6. Get rid of recursion-marker values in enum AlterTableType

  7. Release cache tuple when no longer needed

  8. Empty search_path in logical replication apply worker and walsender.

  9. Refactor format_type APIs to be more modular

  10. Use wrappers of PG_DETOAST_DATUM_PACKED() more.

чт, 25 июл. 2024 г. в 12:35, Amit Kapila <amit.kapila16@gmail.com>:

> On Thu, Mar 28, 2024 at 5:31 PM Andrey M. Borodin <x4mmm@yandex-team.ru>
> wrote:
> >
> > This thread is registered on CF [0] but is not active since 2023. Is
> anyone working on this? I understand that this is a nice feature. Should we
> move it to next CF or withdraw CF entry?
> >
>
> At this stage, we should close either Returned With Feedback or
> Withdrawn as this requires a lot of work.
>
> --
> With Regards,
> Amit Kapila.
>
>
>
>
>
Hello there,
I'm interested in logical DDL replication and I've read through this
thread, which has provided me with a lot of valuable information. However,
I have a couple of questions that I hope, you could help me with:
1) It seems that a lot of the work done here is simply to extend the
existing functionality to work with JSONB. From a development point of
view, it seems appropriate to separate this into a new discussion and
commit, just to expand the functionality of JSONB in terms of use for
development inside the postgres.
2) inside the timeline, it seems that work is moving towards creating an
MVP, which can then be finalized. As a result, it seems that the most
recent fixes, especially those related to table replication, have been
completed, or at least are not discussed in this section. The discussion
ends suddenly, so I don't quite understand: are we facing some unsolvable
problems, or we just didn't have enough time and energy to bring this to an
end?
3) Unfortunately, I couldn't find any specific tests specifically for ddl
logical replication. It seems logical to me that covering all possible
cases of ddl replication with tests will help move the work forward and
convince possible skeptics about worked issues. Did I miss this work or
were they just not implemented for some other reason?
4) We decided to use event trigger and logical_message instead of extending
the standard logical replication functionality by iterating through WAL
records in walsender and decoding there. Was there any reason why we didn't
even consider the possibility of doing everything within the structure of
the existing logical replication architecture, simply extending it to work
with ddl?
5) As for event triggers, I am confused by its use in terms of security and
fault tolerance. After reviewing the source code of event triggers, I did
not find any problems, but it seems strange that this technology is not
used inside Postgres. Perhaps there are reasons for this, or is it such a
good technology that it has no problems (or did I just skip this
discussion)?
6) Regarding testing: Have any synthetic tests been performed to measure
the speed of our replication? How much can we increase the size of WAL, and
how does it compare with classical logical and physical replication?

--
With Regards,
Konstantin Berkaev