Re: adding partitioned tables to publications

amit <amitlangote09@gmail.com>

From: Amit Langote <amitlangote09@gmail.com>

To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>

Cc: Rafia Sabih <rafia.pghackers@gmail.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>

Date: 2020-01-23T10:10:05Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Fix RELCACHE_FORCE_RELEASE issue
- 12fb189bfeaf 13.0 landed
Allow publishing partition changes via ancestors
- 83fd4532a721 13.0 landed
Add logical replication support to replicate into partitioned tables
- f1ac27bfda6c 13.0 landed
Refactor code to look up local replication tuple
- d8653f468789 13.0 landed
Some refactoring of logical/worker.c
- 97ee604d9bce 13.0 landed
Prepare to support non-tables in publications
- c314c147c056 13.0 landed
Support adding partitioned tables to publication
- 17b9e7f9fe23 13.0 landed

Attachments

v10-0002-Some-refactoring-of-logical-worker.c.patch (text/plain) patch v10-0002
v10-0003-Add-subscription-support-to-replicate-into-parti.patch (text/plain) patch v10-0003
v10-0004-Publish-partitioned-table-inserts-as-its-own.patch (text/plain) patch v10-0004
v10-0001-Support-adding-partitioned-tables-to-publication.patch (text/plain) patch v10-0001

On Wed, Jan 22, 2020 at 2:38 PM Amit Langote <amitlangote09@gmail.com> wrote:
> Other than that, the updated patch contains following significant changes:
>
> * Changed pg_publication.c: GetPublicationRelations() so that any
> published partitioned tables are expanded as needed
>
> * Since the pg_publication_tables view is backed by
> GetPublicationRelations(), that means subscriptioncmds.c:
> fetch_table_list() no longer needs to craft a query to include
> partitions when needed, because partitions are included at source.
> That seems better, because it allows to limit the complexity
> surrounding publication of partitioned tables to the publication side.
>
> * Fixed the publication table DDL to spot more cases of tables being
> added to a publication in a duplicative manner.  For example,
> partition being added to a publication which already contains its
> ancestor and a partitioned tables being added to a publication
> (implying all of its partitions are added) which already contains a
> partition

On second thought, this seems like an overkill.  It might be OK after
all for both a partitioned table and its partitions to be explicitly
added to a publication without complaining of duplication. IOW, it's
the user's call whether it makes sense to do that or not.

> Only attaching 0001.

Attached updated 0001 considering the above and the rest of the
patches that add support for replicating partitioned tables using
their own identity and schema.  I have reorganized the other patches
as follows:

0002: refactoring of logical/worker.c without any functionality
changes (contains much less churn than in earlier versions)

0003: support logical replication into partitioned tables on the
subscription side (allows replicating from a non-partitioned table on
publisher node into a partitioned table on subscriber node)

0004: support optionally replicating partitioned table changes (and
changes directly made to partitions) using root partitioned table
identity and schema

Thanks,
Amit