Re: Support logical replication of DDLs
li jie <ggysxcq@gmail.com>
From: li jie <ggysxcq@gmail.com>
To: vignesh C <vignesh21@gmail.com>
Cc: Peter Smith <smithpb2250@gmail.com>, Zheng Li <zhengli10@gmail.com>, Ajin Cherian <itsajin@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>,
"houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Amit Kapila <amit.kapila16@gmail.com>,
Masahiko Sawada <sawada.mshk@gmail.com>, Japin Li <japinli@hotmail.com>, rajesh singarapu <rajesh.rs0541@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-11-25T09:06:44Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Add a run_as_owner option to subscriptions.
- 482675987bcd 16.0 cited
-
Refactor pgoutput_change().
- da324d6cd45b 16.0 cited
-
Print the correct aliases for DML target tables in ruleutils.
- df931e9ab35b 11.20 landed
- c8a5f1685fb7 15.3 landed
- 4efb4f0d4878 13.11 landed
- 3dd287c14fac 12.15 landed
- 393430f57544 16.0 landed
- 14345f3c6a7b 14.8 landed
-
Fix object identity string for transforms
- 9a312562314a 16.0 landed
-
Add grantable MAINTAIN privilege and pg_maintain role.
- 60684dd834a2 16.0 cited
-
Get rid of recursion-marker values in enum AlterTableType
- 840ff5f451cd 16.0 cited
-
Release cache tuple when no longer needed
- ed0fbc8e5ac9 15.0 cited
-
Empty search_path in logical replication apply worker and walsender.
- 11da97024abb 14.0 cited
-
Refactor format_type APIs to be more modular
- a26116c6cbf4 11.0 cited
-
Use wrappers of PG_DETOAST_DATUM_PACKED() more.
- 3a0d473192b2 10.0 cited
Hi Developer,
I have been following this patch for a long time.
Recently, I started to try to test it. I found several bugs
here and want to give you feedback.
1. CREATE TABLE LIKE
I found that this case may be repication incorrectly.
You can run the following SQL statement:
```
CREATE TABLE ctlt1 (a text CHECK (length(a) > 2) PRIMARY KEY, b text);
ALTER TABLE ctlt1 ALTER COLUMN a SET STORAGE MAIN;
ALTER TABLE ctlt1 ALTER COLUMN b SET STORAGE EXTERNAL;
CREATE TABLE ctlt1_like (LIKE ctlt1 INCLUDING ALL);
```
The ctlt1_like table will not be able to correct the replication.
I think this is because create table like statement is captured by
the event trigger to a create table statement and multiple alter table
statements.
There are some overlaps between them, and an error is reported when
downstream replication occurs.
2. ALTER TABLE (inherits)
case:
```
CREATE TABLE gtest30 (
a int,
b int GENERATED ALWAYS AS (a * 2) STORED
);
CREATE TABLE gtest30_1 () INHERITS (gtest30);
ALTER TABLE gtest30 ALTER COLUMN b DROP EXPRESSION;
```
After this case is executed in the publication, the following error occurs
in the subscription :
ERROR: column "b" of relation "gtest30" is not a stored generated column
STATEMENT: ALTER TABLE public.gtest30 ALTER COLUMN b DROP EXPRESSION, ALTER
COLUMN b DROP EXPRESSION
Obviously, the column modifications of the inherited table were also
captured,
and then deparse the wrong statement.
I believe that such errors may also occur in other alter table subcmd
scenarios where tables are inherited.
3. ALTER TABLE SET STATISTICS
case:
```
CREATE TABLE test_stat (a int);
ALTER TABLE test_stat ALTER a SET STATISTICS -1;
```
After this case is executed in the publication, the following error occurs
in the subscription :
syntax error at or near "4294967295" at character 60
STATEMENT: ALTER TABLE public.test_stat ALTER COLUMN a SET STATISTICS
4294967295
I guess this should be an overflow in the integer conversion process.
4. json null string coredump
case:
```
CREATE OR REPLACE FUNCTION test_ddl_deparse_full()
RETURNS event_trigger LANGUAGE plpgsql AS
$$
DECLARE
r record;
deparsed_json text;
BEGIN
FOR r IN SELECT * FROM pg_event_trigger_ddl_commands()
LOOP
deparsed_json = ddl_deparse_to_json(r.command);
RAISE NOTICE 'deparsed json: %', deparsed_json;
RAISE NOTICE 're-formed command: %',
ddl_deparse_expand_command(deparsed_json);
END LOOP;
END;
$$;
CREATE EVENT TRIGGER test_ddl_deparse_full
ON ddl_command_end EXECUTE PROCEDURE test_ddl_deparse_full();
CREATE SCHEMA AUTHORIZATION postgres;
```
If the preceding case is executed, coredump occurs,
which is related to null string and can be reproduced.
I hope these feedbacks can be helpful to you.
We sincerely wish you complete the ddl Logical replication feature.
Regards, Adger
vignesh C <vignesh21@gmail.com> 于2022年11月25日周五 14:18写道:
> On Sun, 20 Nov 2022 at 09:29, vignesh C <vignesh21@gmail.com> wrote:
> >
> > On Fri, 11 Nov 2022 at 11:03, Peter Smith <smithpb2250@gmail.com> wrote:
> > >
> > > On Fri, Nov 11, 2022 at 4:17 PM Peter Smith <smithpb2250@gmail.com>
> wrote:
> > > >
> > > > On Fri, Nov 11, 2022 at 4:09 PM Peter Smith <smithpb2250@gmail.com>
> wrote:
> > > > >
> > > > > On Fri, Nov 11, 2022 at 3:47 PM Peter Smith <smithpb2250@gmail.com>
> wrote:
> > > > > >
> > > > > > Here are more review comments for the v32-0001 file ddl_deparse.c
> > > > > >
> > > > > > *** NOTE - my review post became too big, so I split it into
> smaller parts.
> > > > >
> > > >
> > >
> > > THIS IS PART 4 OF 4.
> > >
> > > =======
> > >
> > > src/backend/commands/ddl_deparse.c
> >
> > Thanks for the comments, the attached v39 patch has the changes for the
> same.
>
> One comment:
> While fixing review comments, I found that default syntax is not
> handled for create domain:
> + /*
> + * Verbose syntax
> + *
> + * CREATE DOMAIN %{identity}D AS %{type}T %{not_null}s
> %{constraints}s
> + * %{collation}s
> + */
> + createDomain = new_objtree("CREATE");
> +
> + append_object_object(createDomain,
> + "DOMAIN %{identity}D AS",
> +
> new_objtree_for_qualname_id(TypeRelationId,
> +
> objectId));
> + append_object_object(createDomain,
> + "%{type}T",
> +
> new_objtree_for_type(typForm->typbasetype, typForm->typtypmod));
>
> Regards,
> Vignesh
>
>
>