Re: row filtering for logical replication

Euler Taveira <euler@timbira.com.br>

From: Euler Taveira <euler@timbira.com.br>

To: David Fetter <david@fetter.org>

Cc: pgsql-hackers <pgsql-hackers@postgresql.org>

Date: 2018-03-01T15:41:04Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Release cache tuple when no longer needed
- ed0fbc8e5ac9 15.0 landed
Add some additional tests for row filters in logical replication.
- ceb57afd3ce1 15.0 landed
Fix one of the tests introduced in commit 52e4f0cd47.
- cfb4e209ec15 15.0 landed
Allow specifying row filters for logical replication of tables.
- 52e4f0cd472d 15.0 landed
Move scanint8() to numutils.c
- cfc7191dfea3 15.0 cited
Replace Test::More plans with done_testing
- 549ec201d613 15.0 cited
Reduce relcache access in WAL sender streaming logical changes
- 6ce16088bfed 15.0 cited
Small cleanups related to PUBLICATION framework code
- c9105dd3660f 15.0 cited
Add a view to show the stats of subscription workers.
- 8d74fc96db5f 15.0 cited
Allow publishing the tables of schema.
- 5a2832465fd8 15.0 cited
Doc: improve documentation of CREATE/ALTER SUBSCRIPTION.
- 1882d6cca161 15.0 cited
Add PublicationTable and PublicationRelInfo structs
- 0c6828fa987b 15.0 cited
Remove unused argument "txn" in maybe_send_schema().
- 93d573d86571 15.0 cited
Add prepare API support for streaming transactions in logical replication.
- 63cf61cdeb7b 15.0 cited
Unify PostgresNode's new() and get_new_node() methods
- 201a76183e20 15.0 cited
Use l*_node() family of functions where appropriate
- 2b00db4fb0c7 15.0 cited
Add support for prepared transactions to built-in logical replication.
- a8fd13cab0ba 15.0 cited
Restore the portal-level snapshot after procedure COMMIT/ROLLBACK.
- ef9480509622 11.13 cited
Rename a parse node to be more general
- 91d1f2d30210 14.0 landed
Remove unused column atttypmod from initial tablesync query
- 4ad31bb2ef25 14.0 landed
SEARCH and CYCLE clauses
- 3696a600e229 14.0 cited

2018-02-28 21:47 GMT-03:00 David Fetter <david@fetter.org>:
> I noticed that the WHERE clause applies to all tables in the
> publication.  Is that actually the right thing?  I'm thinking of a
> case where we have foo(id, ...) and bar(foo_id, ....).  To slice that
> correctly, we'd want to do the ids in the foo table and the foo_ids in
> the bar table.  In the system as written, that would entail, at least
> potentially, writing a lot of publications by hand.
>
I didn't make it clear in my previous email and I think you misread
the attached docs. Each table can have an optional WHERE clause. I'll
made it clear when I rewrite the tests. Something like:

CREATE PUBLICATION tap_pub FOR TABLE tab_rowfilter_1 WHERE (a > 1000
AND b <> 'filtered'), tab_rowfilter_2 WHERE (c % 2 = 0),
tab_rowfilter_3;

Such syntax will not block another future feature that will publish
only few columns of the table.

> I also noticed that in psql, \dRp+ doesn't show the WHERE clause,
> which it probably should.
>
Yea, it could be added be I'm afraid of such long WHERE clauses.

> Does it need regression tests?
>
I included some tests just to demonstrate the feature but I'm planning
to add a separate test file for it.


-- 
   Euler Taveira                                   Timbira -
http://www.timbira.com.br/
   PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento