Re: row filtering for logical replication

Erik Rijkers <er@xs4all.nl>

From: Erik Rijkers <er@xs4all.nl>

To: Euler Taveira <euler@timbira.com.br>

Cc: Alexey Zagarin <zagarin@gmail.com>, Andres Freund <andres@anarazel.de>, Petr Jelinek <petr.jelinek@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Suzuki Hironobu <hironobu@interdb.jp>, a.kondratov@postgrespro.ru

Date: 2019-09-03T05:50:29Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Release cache tuple when no longer needed
- ed0fbc8e5ac9 15.0 landed
Add some additional tests for row filters in logical replication.
- ceb57afd3ce1 15.0 landed
Fix one of the tests introduced in commit 52e4f0cd47.
- cfb4e209ec15 15.0 landed
Allow specifying row filters for logical replication of tables.
- 52e4f0cd472d 15.0 landed
Move scanint8() to numutils.c
- cfc7191dfea3 15.0 cited
Replace Test::More plans with done_testing
- 549ec201d613 15.0 cited
Reduce relcache access in WAL sender streaming logical changes
- 6ce16088bfed 15.0 cited
Small cleanups related to PUBLICATION framework code
- c9105dd3660f 15.0 cited
Add a view to show the stats of subscription workers.
- 8d74fc96db5f 15.0 cited
Allow publishing the tables of schema.
- 5a2832465fd8 15.0 cited
Doc: improve documentation of CREATE/ALTER SUBSCRIPTION.
- 1882d6cca161 15.0 cited
Add PublicationTable and PublicationRelInfo structs
- 0c6828fa987b 15.0 cited
Remove unused argument "txn" in maybe_send_schema().
- 93d573d86571 15.0 cited
Add prepare API support for streaming transactions in logical replication.
- 63cf61cdeb7b 15.0 cited
Unify PostgresNode's new() and get_new_node() methods
- 201a76183e20 15.0 cited
Use l*_node() family of functions where appropriate
- 2b00db4fb0c7 15.0 cited
Add support for prepared transactions to built-in logical replication.
- a8fd13cab0ba 15.0 cited
Restore the portal-level snapshot after procedure COMMIT/ROLLBACK.
- ef9480509622 11.13 cited
Rename a parse node to be more general
- 91d1f2d30210 14.0 landed
Remove unused column atttypmod from initial tablesync query
- 4ad31bb2ef25 14.0 landed
SEARCH and CYCLE clauses
- 3696a600e229 14.0 cited

Attachments

logrep_rowfilter_2_nodes.sh (text/x-shellscript)

On 2019-09-03 05:32, Euler Taveira wrote:
> Em ter, 3 de set de 2019 às 00:16, Alexey Zagarin <zagarin@gmail.com> 
> escreveu:
>> 
>> There are complaints in the log (both pub and sub) like:
>> ERROR: trying to store a heap tuple into wrong type of slot
>> 
>> I have no idea what causes that.
>> 
>> Yeah, I've seen that too. It was fixed by Alexey Kondratov, in line 
>> 955 of 0005-Row-filtering-for-logical-replication.patch it should be 
>> &TTSOpsHeapTuple instead of &TTSOpsVirtual.
>> 
> Ops... exact. That was an oversight while poking with different types 
> of slots.

OK, I'll consider Alexey Kondratov's set of patches as the current 
state-of-the-art then.  (They still apply.)

I found a problem where I'm not sure it's a bug:

The attached bash script does a test by setting up pgbench tables on 
both master and replica, and then sets up logical replication for a 
slice of pgbench_accounts. Then it does a short pgbench run, and loops 
until the results become identical(ok) (or breaks out after a certain 
time (NOK=not ok)).

It turns out this did not work until I added a wait state after the 
CREATE SUBSCRIPTION.  It always fails without the wait state, and always 
works with the wait state.

Do you agree this is a bug?

thanks (also to both Alexeys :))

Erik Rijkers

PS
by the way, this script won't run as-is on other machines; it has stuff 
particular to my local setup.