Re: row filtering for logical replication

Euler Taveira <euler@eulerto.com>

From: "Euler Taveira" <euler@eulerto.com>

To: Önder Kalacı <onderkalaci@gmail.com>

Cc: japin <japinli@hotmail.com>, "Michael Paquier" <michael@paquier.xyz>, "David Steele" <david@pgmasters.net>, "Craig Ringer" <craig@2ndquadrant.com>, "Tomas Vondra" <tomas.vondra@2ndquadrant.com>, "Amit Langote" <amitlangote09@gmail.com>, "PostgreSQL Hackers" <pgsql-hackers@lists.postgresql.org>

Date: 2021-02-27T01:35:13Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Release cache tuple when no longer needed
- ed0fbc8e5ac9 15.0 landed
Add some additional tests for row filters in logical replication.
- ceb57afd3ce1 15.0 landed
Fix one of the tests introduced in commit 52e4f0cd47.
- cfb4e209ec15 15.0 landed
Allow specifying row filters for logical replication of tables.
- 52e4f0cd472d 15.0 landed
Move scanint8() to numutils.c
- cfc7191dfea3 15.0 cited
Replace Test::More plans with done_testing
- 549ec201d613 15.0 cited
Reduce relcache access in WAL sender streaming logical changes
- 6ce16088bfed 15.0 cited
Small cleanups related to PUBLICATION framework code
- c9105dd3660f 15.0 cited
Add a view to show the stats of subscription workers.
- 8d74fc96db5f 15.0 cited
Allow publishing the tables of schema.
- 5a2832465fd8 15.0 cited
Doc: improve documentation of CREATE/ALTER SUBSCRIPTION.
- 1882d6cca161 15.0 cited
Add PublicationTable and PublicationRelInfo structs
- 0c6828fa987b 15.0 cited
Remove unused argument "txn" in maybe_send_schema().
- 93d573d86571 15.0 cited
Add prepare API support for streaming transactions in logical replication.
- 63cf61cdeb7b 15.0 cited
Unify PostgresNode's new() and get_new_node() methods
- 201a76183e20 15.0 cited
Use l*_node() family of functions where appropriate
- 2b00db4fb0c7 15.0 cited
Add support for prepared transactions to built-in logical replication.
- a8fd13cab0ba 15.0 cited
Restore the portal-level snapshot after procedure COMMIT/ROLLBACK.
- ef9480509622 11.13 cited
Rename a parse node to be more general
- 91d1f2d30210 14.0 landed
Remove unused column atttypmod from initial tablesync query
- 4ad31bb2ef25 14.0 landed
SEARCH and CYCLE clauses
- 3696a600e229 14.0 cited

Attachments

0001-Rename-a-WHERE-node.patch (text/x-patch) patch 0001
0002-Row-filter-for-logical-replication.patch (text/x-patch) patch 0002
0003-Print-publication-WHERE-condition-in-psql.patch (text/x-patch) patch 0003
0004-Publication-WHERE-condition-support-for-pg_dump.patch (text/x-patch) patch 0004
0005-Measure-row-filter-overhead.patch (text/x-patch) patch 0005
0006-Debug-messages.patch (text/x-patch) patch 0006

On Mon, Feb 22, 2021, at 9:28 AM, Euler Taveira wrote:
> On Mon, Feb 22, 2021, at 7:45 AM, Önder Kalacı wrote:
>> Thanks for working on this. I did some review and testing, please see my comments below.
> I appreciate your review. I'm working on a new patch set and expect to post it soon.
I'm attaching a new patch set. This new version improves documentation and commit  
messages and incorporates a few debug messages. I did a couple of tests and    
didn't find issues.

Here are some numbers from my i7 using a simple expression (aid > 0) on table
pgbench_accounts.

$ awk '/row filter time/ {print $9}' postgresql.log | /tmp/stat.pl 99
mean:           33.00 us
stddev:         17.65 us
median:         28.83 us
min-max:        [3.48 .. 6404.84] us
percentile(99): 49.66 us
mode:           41.71 us

I don't expect 0005 and 0006 to be included. I attached them to help with some
tests.


--
Euler Taveira
EDB   https://www.enterprisedb.com/