Re: row filtering for logical replication

Tomas Vondra <tomas.vondra@2ndquadrant.com>

From: Tomas Vondra <tomas.vondra@2ndquadrant.com>

To: Petr Jelinek <petr.jelinek@2ndquadrant.com>, fabriziomello@gmail.com

Cc: Euler Taveira de Oliveira <euler@timbira.com.br>, Pgsql Hackers <pgsql-hackers@postgresql.org>, hironobu@interdb.jp

Date: 2018-11-23T20:02:15Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Release cache tuple when no longer needed
- ed0fbc8e5ac9 15.0 landed
Add some additional tests for row filters in logical replication.
- ceb57afd3ce1 15.0 landed
Fix one of the tests introduced in commit 52e4f0cd47.
- cfb4e209ec15 15.0 landed
Allow specifying row filters for logical replication of tables.
- 52e4f0cd472d 15.0 landed
Move scanint8() to numutils.c
- cfc7191dfea3 15.0 cited
Replace Test::More plans with done_testing
- 549ec201d613 15.0 cited
Reduce relcache access in WAL sender streaming logical changes
- 6ce16088bfed 15.0 cited
Small cleanups related to PUBLICATION framework code
- c9105dd3660f 15.0 cited
Add a view to show the stats of subscription workers.
- 8d74fc96db5f 15.0 cited
Allow publishing the tables of schema.
- 5a2832465fd8 15.0 cited
Doc: improve documentation of CREATE/ALTER SUBSCRIPTION.
- 1882d6cca161 15.0 cited
Add PublicationTable and PublicationRelInfo structs
- 0c6828fa987b 15.0 cited
Remove unused argument "txn" in maybe_send_schema().
- 93d573d86571 15.0 cited
Add prepare API support for streaming transactions in logical replication.
- 63cf61cdeb7b 15.0 cited
Unify PostgresNode's new() and get_new_node() methods
- 201a76183e20 15.0 cited
Use l*_node() family of functions where appropriate
- 2b00db4fb0c7 15.0 cited
Add support for prepared transactions to built-in logical replication.
- a8fd13cab0ba 15.0 cited
Restore the portal-level snapshot after procedure COMMIT/ROLLBACK.
- ef9480509622 11.13 cited
Rename a parse node to be more general
- 91d1f2d30210 14.0 landed
Remove unused column atttypmod from initial tablesync query
- 4ad31bb2ef25 14.0 landed
SEARCH and CYCLE clauses
- 3696a600e229 14.0 cited


On 11/23/18 8:14 PM, Petr Jelinek wrote:
> On 23/11/2018 19:29, Fabrízio de Royes Mello wrote:
>>
>> On Fri, Nov 23, 2018 at 4:13 PM Petr Jelinek
>> <petr.jelinek@2ndquadrant.com <mailto:petr.jelinek@2ndquadrant.com>> wrote:
>>>
>>>>
>>>> If carefully documented I see no problem with it... we already have an
>>>> analogous problem with functional indexes.
>>>
>>> The difference is that with functional indexes you can recreate the
>>> missing object and everything is okay again. With logical replication
>>> recreating the object will not help.
>>>
>>
>> In this case with logical replication you should rsync the object. That
>> is the price of misunderstanding / bad use of the new feature.
>>
>> As usual, there are no free beer ;-)
>>
> 
> Yeah but you have to resync whole subscription, not just single table
> (removing table from the publication will also not help), that's pretty
> severe punishment. What if you have triggers downstream that do
> calculations or logging which you can't recover by simply rebuilding
> replica? I think it's better to err on the side of no data loss.
> 

Yeah, having to resync everything because you accidentally dropped a
function is quite annoying. Of course, you should notice that while
testing the upgrade in a testing environment, but still ...

> We could also try to figure out a way to recover from this that does not
> require resync, ie perhaps we could somehow temporarily force evaluation
> of the expression to have current snapshot.
> 

That seems like huge a can of worms ...


cheers

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services