Re: Support logical replication of DDLs
shveta malik <shveta.malik@gmail.com>
From: shveta malik <shveta.malik@gmail.com>
To: "Yu Shi (Fujitsu)" <shiy.fnst@fujitsu.com>
Cc: vignesh C <vignesh21@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>,
Ajin Cherian <itsajin@gmail.com>, "Wei Wang (Fujitsu)" <wangw.fnst@fujitsu.com>,
Runqi Tian <runqidev@gmail.com>, Peter Smith <smithpb2250@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>,
li jie <ggysxcq@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>,
Alvaro Herrera <alvherre@alvh.no-ip.org>, Masahiko Sawada <sawada.mshk@gmail.com>, Japin Li <japinli@hotmail.com>, rajesh singarapu <rajesh.rs0541@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
Zheng Li <zhengli10@gmail.com>, shveta malik <shveta.malik@gmail.com>
Date: 2023-05-31T09:40:34Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Add a run_as_owner option to subscriptions.
- 482675987bcd 16.0 cited
-
Refactor pgoutput_change().
- da324d6cd45b 16.0 cited
-
Print the correct aliases for DML target tables in ruleutils.
- df931e9ab35b 11.20 landed
- c8a5f1685fb7 15.3 landed
- 4efb4f0d4878 13.11 landed
- 3dd287c14fac 12.15 landed
- 393430f57544 16.0 landed
- 14345f3c6a7b 14.8 landed
-
Fix object identity string for transforms
- 9a312562314a 16.0 landed
-
Add grantable MAINTAIN privilege and pg_maintain role.
- 60684dd834a2 16.0 cited
-
Get rid of recursion-marker values in enum AlterTableType
- 840ff5f451cd 16.0 cited
-
Release cache tuple when no longer needed
- ed0fbc8e5ac9 15.0 cited
-
Empty search_path in logical replication apply worker and walsender.
- 11da97024abb 14.0 cited
-
Refactor format_type APIs to be more modular
- a26116c6cbf4 11.0 cited
-
Use wrappers of PG_DETOAST_DATUM_PACKED() more.
- 3a0d473192b2 10.0 cited
Attachments
- 0001-Deparser-for-Table-DDL-commands-and-exten-2023_05_31.patch (application/octet-stream) patch 0001
- 0002-Enhance-the-event-trigger-to-support-DDL--2023_05_31.patch (application/octet-stream) patch 0002
- 0003-Add-verbose-option-for-ddl-deparse-module-2023_05_31.patch (application/octet-stream) patch 0003
- 0005-DDL-replication-for-Table-DDL-commands-2023_05_31.patch (application/octet-stream) patch 0005
- 0004-Introduce-the-test_ddl_deparse_regress-te-2023_05_31.patch (application/octet-stream) patch 0004
- 0006-Add-subscription-tap-test-for-DDL-replica-2023_05_31.patch (application/octet-stream) patch 0006
- 0007-Apply-the-DDL-change-as-that-same-user-th-2023_05_31.patch (application/octet-stream) patch 0007
- 0008-ObjTree-Removal-for-multiple-commands-2023_05_31.patch (application/octet-stream) patch 0008
On Mon, May 29, 2023 at 11:45 AM Yu Shi (Fujitsu) <shiy.fnst@fujitsu.com> wrote:
>
>
>
> Thanks for updating the patch. Here are some comments.
>
Thanks Shi-san for the review.
> 0001 patch
> -----
> 1.
> + colname = get_attname(ownerId, depform->refobjsubid, false);
> + if (colname == NULL)
> + continue;
>
> missing_ok is false when calling get_attname(), so is there any case that
> colname is NULL?
>
Removed this check in patch 0008 .
> 2.
> + case AT_SetStatistics:
> + {
> + Assert(IsA(subcmd->def, Integer));
> + if (subcmd->name)
> + tmp_obj = new_objtree_VA("ALTER COLUMN %{column}I SET STATISTICS %{statistics}n", 3,
> + "type", ObjTypeString, "set statistics",
> + "column", ObjTypeString, subcmd->name,
> + "statistics", ObjTypeInteger,
> + intVal((Integer *) subcmd->def));
> + else
> + tmp_obj = new_objtree_VA("ALTER COLUMN %{column}n SET STATISTICS %{statistics}n", 3,
> + "type", ObjTypeString, "set statistics",
> + "column", ObjTypeInteger, subcmd->num,
> + "statistics", ObjTypeInteger,
> + intVal((Integer *) subcmd->def));
> + subcmds = lappend(subcmds, new_object_object(tmp_obj));
> + }
> + break;
>
> I think subcmd->name will be NULL only if relation type is index. So should it
> be removed because currently only table commands are supported?
>
Removed this check in patch 0008
> 0002 patch
> -----
> 3.
> + /* Skip adding constraint for inherits table sub command */
> + if (!constrOid)
> + continue;
>
> Would it be better to use OidIsValid() here?
>
yes, modified in patch 0008
> 0008 patch
> -----
> 4.
> case AT_AddColumn:
> /* XXX need to set the "recurse" bit somewhere? */
> Assert(IsA(subcmd->def, ColumnDef));
> - tree = deparse_ColumnDef(rel, dpcontext, false,
> - (ColumnDef *) subcmd->def, true, &expr);
>
> mark_function_volatile(context, expr);
>
> After this change, `expr` is not assigned a value when mark_function_volatile is called.
>
Corrected.
> Some problems I saw :
> -----
> 5.
> create table p1(f1 int);
> create table p1_c1() inherits(p1);
> alter table p1 add constraint inh_check_constraint1 check (f1 > 0);
> alter table p1_c1 add constraint inh_check_constraint1 check (f1 > 0);
>
> The re-formed command of the last command is "ALTER TABLE public.p1_c1", which
> seems to be wrong.
>
Fixed, second alter-table should actually be no-op in terms of
deparsing. But when it is run without running the first alter-table
command, it should generate the reformed command.
> 6.
> SET allow_in_place_tablespaces = true;
> CREATE TABLESPACE ddl_tblspace LOCATION '';
> RESET allow_in_place_tablespaces;
> CREATE TABLE tbl_index_tblspe (a int, PRIMARY KEY(a) USING INDEX TABLESPACE ddl_tblspace) ;
>
> The re-formed command of the last command seems incorrect:
> CREATE TABLE public.tbl_index_tblspe (a pg_catalog.int4 STORAGE PLAIN , USING INDEX TABLESPACE ddl_tblspace)
>
Fixed.
> 7.
> CREATE TABLE part2_with_multiple_storage_params(
> id int,
> name varchar
> ) WITH (autovacuum_enabled);
>
> re-formed command: CREATE TABLE public.part2_with_multiple_storage_params (id pg_catalog.int4 STORAGE PLAIN , name pg_catalog."varchar" STORAGE EXTENDED COLLATE pg_catalog."default" ) WITH (vacuum_index_cleanup = 'on', autovacuum_vacuum_scale_factor = '0.2', vacuum_truncate = 'true', autovacuum_enabled = 'TRUE')
>
> When the option is not specified, re-formed command used uppercase letters. The
> reloptions column in pg_class of the original command is
> "{autovacuum_enabled=true}", but that of the re-formed command is
> "{autovacuum_enabled=TRUE}". I tried to add this case to
> test_ddl_deparse_regress test module but the test failed because the dumped
> results are different.
>
Changed to lowercase for the sake of tests.
PFA the set of patches consisting above changes. All the changes are
made in 0008 patch.
Apart from above changes, many partition attach/detach related tests
are uncommented in alter_table.sql in patch 0008.
thanks
Shveta