Re: Support logical replication of DDLs
vignesh C <vignesh21@gmail.com>
From: vignesh C <vignesh21@gmail.com>
To: li jie <ggysxcq@gmail.com>
Cc: Ajin Cherian <itsajin@gmail.com>,
rajesh singarapu <rajesh.rs0541@gmail.com>, Zheng Li <zhengli10@gmail.com>, Peter Smith <smithpb2250@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>,
Alvaro Herrera <alvherre@alvh.no-ip.org>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>,
Amit Kapila <amit.kapila16@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, Japin Li <japinli@hotmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-11-29T18:31:49Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Add a run_as_owner option to subscriptions.
- 482675987bcd 16.0 cited
-
Refactor pgoutput_change().
- da324d6cd45b 16.0 cited
-
Print the correct aliases for DML target tables in ruleutils.
- df931e9ab35b 11.20 landed
- c8a5f1685fb7 15.3 landed
- 4efb4f0d4878 13.11 landed
- 3dd287c14fac 12.15 landed
- 393430f57544 16.0 landed
- 14345f3c6a7b 14.8 landed
-
Fix object identity string for transforms
- 9a312562314a 16.0 landed
-
Add grantable MAINTAIN privilege and pg_maintain role.
- 60684dd834a2 16.0 cited
-
Get rid of recursion-marker values in enum AlterTableType
- 840ff5f451cd 16.0 cited
-
Release cache tuple when no longer needed
- ed0fbc8e5ac9 15.0 cited
-
Empty search_path in logical replication apply worker and walsender.
- 11da97024abb 14.0 cited
-
Refactor format_type APIs to be more modular
- a26116c6cbf4 11.0 cited
-
Use wrappers of PG_DETOAST_DATUM_PACKED() more.
- 3a0d473192b2 10.0 cited
Attachments
- v41-0002-Support-DDL-replication.patch (text/x-patch) patch v41-0002
- v41-0003-Support-CREATE-TABLE-AS-SELECT-INTO.patch (text/x-patch) patch v41-0003
- v41-0004-Test-cases-for-DDL-replication.patch (text/x-patch) patch v41-0004
- v41-0001-Functions-to-deparse-DDL-commands.patch (text/x-patch) patch v41-0001
- v41-0005-Do-not-generate-WAL-log-for-non-top-level-DDL-co.patch (text/x-patch) patch v41-0005
On Tue, 29 Nov 2022 at 17:51, li jie <ggysxcq@gmail.com> wrote: > > I will continue to give feedback for this patch. > > 1. LIKE STORAGE > ``` > CREATE TABLE ctlt (a text, c text); > ALTER TABLE ctlt ALTER COLUMN c SET STORAGE EXTERNAL; > CREATE TABLE ctlt_storage (LIKE ctlt INCLUDING STORAGE); > ``` > > postgres=# \d+ ctlt_storage > > Table "public.ctlt_storage" > > Column | Type | Collation | Nullable | Default | Storage | > Compression | Stats target | Description > > --------+------+-----------+----------+---------+----------+-------------+--------------+------------- > > a | text | | | | extended | > | | > > c | text | | | | extended | > | | > > > It can be seen that the storage attribute in column C of table > ctlt_storage is not replicated. > > After the CREATE TABLE LIKE statement is converted, > the LIKE STORAGE attribute is lost because it is difficult to display > it in the CREATE TABLE syntax. > Maybe we need to add a statement to it, like 'ALTER TABLE ctlt_storage > ALTER COLUMN c SET STORAGE EXTERNAL;'. This is fixed with the attached patch. > 2. Reference subcommand be dropped. > ``` > create table another (f1 int, f2 text, f3 text); > > alter table another > alter f1 type text using f2 || ' and ' || f3 || ' more', > alter f2 type bigint using f1 * 10, > drop column f3; > ``` > > The following error occurs downstream: > ERROR: column "?dropped?column?" does not exist at character 206 > STATEMENT: ALTER TABLE public.another DROP COLUMN f3 , ALTER COLUMN > f1 SET DATA TYPE pg_catalog.text COLLATE pg_catalog."default" USING > (((f2 OPERATOR(pg_catalog.||) ' and '::pg_catalog.text) > OPERATOR(pg_catalog.||) "?dropped?column?") OPERATOR(pg_catalog.||) ' > more'::pg_catalog.text), ALTER COLUMN f2 SET DATA TYPE pg_catalog.int8 > USING (f1 OPERATOR(pg_catalog.*) 10) > > Obviously, column f3 has been deleted and its name no longer exists. > Maybe we need to keep it and save it in advance like a drop object. > However, ATLER TABLE is complex, and this problem also occurs in > other similar scenarios. This is slightly tricky, we will fix this in the next version. Also a couple of other issues reported are fixed in this patch: 3. ALTER TABLE SET STATISTICS CREATE TABLE test_stat (a int); ALTER TABLE test_stat ALTER a SET STATISTICS -1; 4. json null string coredump CREATE OR REPLACE FUNCTION test_ddl_deparse_full() RETURNS event_trigger LANGUAGE plpgsql AS $$ DECLARE r record; deparsed_json text; BEGIN FOR r IN SELECT * FROM pg_event_trigger_ddl_commands() LOOP deparsed_json = ddl_deparse_to_json(r.command); RAISE NOTICE 'deparsed json: %', deparsed_json; RAISE NOTICE 're-formed command: %', ddl_deparse_expand_command(deparsed_json); END LOOP; END; $$; CREATE EVENT TRIGGER test_ddl_deparse_full ON ddl_command_end EXECUTE PROCEDURE test_ddl_deparse_full(); CREATE SCHEMA AUTHORIZATION postgres; The attached v41 patch has the fixes for the above 3 issues. Regards, Vignesh