Re: Logical Replication of sequences
vignesh C <vignesh21@gmail.com>
Commits
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Doc: Add documentation for sequence synchronization.
- 55cefadde874 19 (unreleased) landed
-
Remove unused assignment in CREATE PUBLICATION grammar.
- bfb7419b0bbe 19 (unreleased) landed
-
Add seq_sync_error_count to subscription statistics.
- f6a4c498dcf6 19 (unreleased) landed
-
Fix few issues in commit 5509055d69.
- 5a4eba558aa7 19 (unreleased) landed
-
Add sequence synchronization for logical replication.
- 5509055d6956 19 (unreleased) landed
-
Add worker type argument to logical replication worker functions.
- 3e8e05596a02 19 (unreleased) landed
-
Introduce "REFRESH SEQUENCES" for subscriptions.
- f0b3573c3aac 19 (unreleased) landed
-
Refactor logical worker synchronization code into a separate file.
- 41c674d2e31e 19 (unreleased) landed
-
Standardize use of REFRESH PUBLICATION in code and messages.
- 2436b8c047ff 19 (unreleased) landed
-
Add "ALL SEQUENCES" support to publications.
- 96b378497346 19 (unreleased) landed
-
Expose sequence page LSN via pg_get_sequence_data.
- b93172ca59f4 19 (unreleased) landed
-
Resume conflict-relevant data retention automatically.
- 0d48d393d465 19 (unreleased) cited
-
Post-commit review fixes for 228c370868.
- 1f7e9ba3ac4e 19 (unreleased) cited
-
Generate GUC tables from .dat file
- 63599896545c 19 (unreleased) cited
Attachments
- v20250204-0001-Introduce-pg_sequence_state-function-for-e.patch (text/x-patch) patch v20250204-0001
- v20250204-0002-Introduce-ALL-SEQUENCES-support-for-Postgr.patch (text/x-patch) patch v20250204-0002
- v20250204-0005-Documentation-for-sequence-synchronization.patch (text/x-patch) patch v20250204-0005
- v20250204-0003-Reorganize-tablesync-Code-and-Introduce-sy.patch (text/x-patch) patch v20250204-0003
- v20250204-0004-Enhance-sequence-synchronization-during-su.patch (text/x-patch) patch v20250204-0004
On Fri, 3 Jan 2025 at 06:53, Peter Smith <smithpb2250@gmail.com> wrote:
>
> Hi Vignesh.
>
> Here are some review comments for patch v20241230-0002
>
> ======
> 1. SYNTAX
>
> The proposed syntax is currently:
>
> CREATE PUBLICATION name
> [ FOR ALL object_type [, ...]
> | FOR publication_object [, ... ] ]
> [ WITH ( publication_parameter [= value] [, ... ] ) ]
>
> where object_type is one of:
>
> TABLES
> SEQUENCES
>
> where publication_object is one of:
>
> TABLE [ ONLY ] table_name [ * ] [ ( column_name [, ... ] ) ] [
> WHERE ( expression ) ] [, ... ]
> TABLES IN SCHEMA { schema_name | CURRENT_SCHEMA } [, ... ]
> ~
>
> But lately, I've been thinking it could be clearer if you removed the
> object_type and instead fully spelled out FOR ALL TABLES and/or FOR
> ALL SEQUENCES.
>
> compare
> CREATE PUBLICATION FOR ALL TABLES, SEQUENCES;
> versus
> CREATE PUBLICATION FOR ALL TABLES, ALL SEQUENCES;
>
> ~
>
> Also AFAICT, the current syntax says it is impossible to mix FOR ALL
> SEQUENCES with FOR TABLE etc but really that *should* be allowed,
> right?
>
> And it looks like you may come to similar grief in future if you try
> things like:
> "FOR ALL TABLES" mixed with "FOR SEQUENCE seq_name"
> "FOR ALL TABLES" mixed with "FOR SEQUENCES IN SCHEMA schema_name"
>
> ~
>
> So, maybe a revised syntax like below would end up being easier and
> also more flexible:
>
> CREATE PUBLICATION name
> [ FOR publication_object [, ... ] ]
> [ WITH ( publication_parameter [= value] [, ... ] ) ]
>
> where publication_object is one of:
>
> ALL TABLES
> ALL SEQUENCES
> TABLE [ ONLY ] table_name [ * ] [ ( column_name [, ... ] ) ] [
> WHERE ( expression ) ] [, ... ]
> TABLES IN SCHEMA { schema_name | CURRENT_SCHEMA } [, ... ]
The proposed be more easier to extend the syntax in the future, so
modified.The proposed be more easier to extend the syntax in the
future, so modified.
> ======
> src/backend/commands/publicationcmds.c
>
> CreatePublication:
>
> 2.
> - /* FOR ALL TABLES requires superuser */
> - if (stmt->for_all_tables && !superuser())
> + /* FOR ALL TABLES or FOR ALL SEQUENCES requires superuser */
> + if ((stmt->for_all_tables || stmt->for_all_sequences) && !superuser())
> ereport(ERROR,
> (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
> - errmsg("must be superuser to create FOR ALL TABLES publication")));
> + errmsg("must be superuser to create ALL TABLES and/or ALL SEQUENCES
> publication")));
>
>
> 2a.
> Typo.
>
> /create ALL TABLES and/or ALL SEQUENCES publication/create a FOR ALL
> TABLES and/or a FOR ALL SEQUENCES publication/
Modified
> ~
>
> 2b.
> This message might be OK now, but I suspect it will become very messy
> in future after you introduce another syntax like "FOR SEQUENCE
> seq_name" etc (which would also be able to be used in combination with
> a FOR ALL TABLES).
>
> So, I think that for future-proofing against all the possible (future)
> combinations, and for keeping the code cleaner, it will be far simpler
> to just keep the errors for tables and sequences separated:
>
> SUGGESTION:
> if (!superuser())
> {
> if (stmt->for_all_tables)
> ereport(ERROR, ... FOR ALL TABLES ...);
> if (stmt->for_all_sequences)
> ereport(ERROR, ... FOR ALL SEQUENCES ...);
> }
If we do that way it will not print both the stmt publication type if
both "ALL TABLES" and "ALL SEQUENCES" is specified.
> ~~~
>
> AlterPublicationOwner_internal:
>
> 3.
> - if (form->puballtables && !superuser_arg(newOwnerId))
> + /* FOR ALL TABLES or FOR ALL SEQUENCES requires superuser */
> + if ((form->puballtables || form->puballsequences) &&
> + !superuser_arg(newOwnerId))
> ereport(ERROR,
> (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
> errmsg("permission denied to change owner of publication \"%s\"",
> NameStr(form->pubname)),
> - errhint("The owner of a FOR ALL TABLES publication must be a superuser.")));
> + errhint("The owner of ALL TABLES and/or ALL SEQUENCES publication
> must be a superuser.")));
>
> Ditto the above comment #2.
Modified the message
> ======
> src/bin/psql/describe.c
>
> 4.
> + puboid_col = cols++;
> + pubname_col = cols++;
> + pubowner_col = cols++;
> + puballtables_col = cols++;
> +
> + if (has_pubsequence)
> + {
> + appendPQExpBufferStr(&buf,
> + ", puballsequences");
> + puballsequences_col = cols++;
> + }
> +
> + appendPQExpBufferStr(&buf,
> + ", pubinsert, pubupdate, pubdelete");
> + pubins_col = cols++;
> + pubupd_col = cols++;
> + pubdel_col = cols++;
> +
> if (has_pubtruncate)
> + {
> appendPQExpBufferStr(&buf,
> ", pubtruncate");
> + pubtrunc_col = cols++;
> + }
> +
> if (has_pubgencols)
> + {
> appendPQExpBufferStr(&buf,
> ", pubgencols");
> + pubgen_col = cols++;
> + }
> +
> if (has_pubviaroot)
> + {
> appendPQExpBufferStr(&buf,
> ", pubviaroot");
> + pubviaroot_col = cols++;
> + }
>
> There is some overlap/duplication of the new variable 'cols' and the
> existing variable 'ncols'.
>
> AFAICT you can just move/replace the declaration of 'ncols' to where
> 'cols' is declared, and then you can remove the duplicated code below
> (because the above code is already doing the same thing).
>
> if (has_pubtruncate)
> ncols++;
> if (has_pubgencols)
> ncols++;
> if (has_pubviaroot)
> ncols++;
> if (has_pubsequence)
> ncols++;
I have removed ncols and used cols.
The attached patch has the changes for the same.
Regards,
Vignesh