Horton

Re: row filtering for logical replication

Erik Rijkers <er@xs4all.nl>

View thread
From: Erik Rijkers <er@xs4all.nl>
To: Euler Taveira <euler@timbira.com.br>
Cc: Andres Freund <andres@anarazel.de>, Petr Jelinek <petr.jelinek@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Suzuki Hironobu <hironobu@interdb.jp>, a.kondratov@postgrespro.ru
Date: 2019-09-02T23:08:25Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Release cache tuple when no longer needed

  2. Add some additional tests for row filters in logical replication.

  3. Fix one of the tests introduced in commit 52e4f0cd47.

  4. Allow specifying row filters for logical replication of tables.

  5. Move scanint8() to numutils.c

  6. Replace Test::More plans with done_testing

  7. Reduce relcache access in WAL sender streaming logical changes

  8. Small cleanups related to PUBLICATION framework code

  9. Add a view to show the stats of subscription workers.

  10. Allow publishing the tables of schema.

  11. Doc: improve documentation of CREATE/ALTER SUBSCRIPTION.

  12. Add PublicationTable and PublicationRelInfo structs

  13. Remove unused argument "txn" in maybe_send_schema().

  14. Add prepare API support for streaming transactions in logical replication.

  15. Unify PostgresNode's new() and get_new_node() methods

  16. Use l*_node() family of functions where appropriate

  17. Add support for prepared transactions to built-in logical replication.

  18. Restore the portal-level snapshot after procedure COMMIT/ROLLBACK.

  19. Rename a parse node to be more general

  20. Remove unused column atttypmod from initial tablesync query

  21. SEARCH and CYCLE clauses

Attachments

On 2019-09-02 01:43, Euler Taveira wrote:
> Em dom, 1 de set de 2019 às 06:09, Erik Rijkers <er@xs4all.nl> 
> escreveu:
>> 
>> The first 4 of these apply without error, but I can't get 0005 to 
>> apply.
>> This is what I use:
>> 
> Erik, I generate a new patch set with patience diff algorithm. It
> seems it applies cleanly.
> 

It did apply cleanly, thanks.

But I can't get it to correctly do the partial replication in the 
attached pgbench-script (similar versions of which script I also used 
for earlier versions of the patch, last year).

There are complaints in the log (both pub and sub) like:
ERROR:  trying to store a heap tuple into wrong type of slot

I have no idea what causes that.

I attach a zip:

$ unzip -l logrep_rowfilter.zip
Archive:  logrep_rowfilter.zip
   Length      Date    Time    Name
---------  ---------- -----   ----
     17942  2019-09-03 00:47   logfile.6525
     10412  2019-09-03 00:47   logfile.6526
      6913  2019-09-03 00:47   logrep_rowfilter_2_nodes.sh
      3371  2019-09-03 00:47   output.txt
---------                     -------
     38638                     4 files

That bash script runs 2 instances (as compiled on my local setup so it 
will not run as-is) and tries for one minute to get a slice of the 
pgbench_accounts table replicated.  One minute is short but I wanted 
short logfiles; I have tried the same up to 20 minutes without the 
replication completing.  I'll try even longer but in the meantime I hope 
you can figure out why these errors occur.


thanks,


Erik Rijkers