Re: Logical replication timeout problem
Amit Kapila <amit.kapila16@gmail.com>
From: Amit Kapila <amit.kapila16@gmail.com>
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: "wangw.fnst@fujitsu.com" <wangw.fnst@fujitsu.com>,
Fabrice Chapuis <fabrice636861@gmail.com>, Euler Taveira <euler@eulerto.com>, Masahiko Sawada <sawada.mshk@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>,
Peter Smith <smithpb2250@gmail.com>, Simon Riggs <simon.riggs@enterprisedb.com>,
Petr Jelinek <petr.jelinek@enterprisedb.com>, "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
Ajin Cherian <itsajin@gmail.com>
Date: 2023-01-18T12:30:12Z
Lists: pgsql-hackers
On Wed, Jan 18, 2023 at 5:37 PM Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> wrote: > > On Wed, Jan 18, 2023 at 1:49 PM wangw.fnst@fujitsu.com > <wangw.fnst@fujitsu.com> wrote: > > > > On Wed, Jan 18, 2023 at 13:29 PM Amit Kapila <amit.kapila16@gmail.com> wrote: > > > On Tue, Jan 17, 2023 at 6:41 PM Ashutosh Bapat > > > <ashutosh.bapat.oss@gmail.com> wrote: > > > > > > > > On Tue, Jan 17, 2023 at 3:34 PM Amit Kapila <amit.kapila16@gmail.com> wrote: > > > > > > > > > > > > > > > > I am a bit worried about the indirections that the wrappers and hooks > > > > > > create. Output plugins call OutputPluginUpdateProgress() in callbacks > > > > > > but I don't see why ReorderBufferProcessTXN() needs a callback to > > > > > > call OutputPluginUpdateProgress. > > > > > > > > > > > > > > > > Yeah, I think we can do it as we are doing the previous approach but > > > > > we need an additional wrapper (update_progress_cb_wrapper()) as the > > > > > current patch has so that we can add error context information. This > > > > > is similar to why we have a wrapper for all other callbacks like > > > > > change_cb_wrapper. > > > > > > > > > > > > > Ultimately OutputPluginUpdateProgress() will be called - which in turn > > > > will call ctx->update_progress. > > > > > > > > > > No, update_progress_cb_wrapper() should directly call > > > ctx->update_progress(). The key reason to have a > > > update_progress_cb_wrapper() is that it allows us to add error context > > > information (see the usage of output_plugin_error_callback). > > > > I think it makes sense. This also avoids the need for every output plugin to > > implement the callback. So I tried to improve the patch based on this approach. > > > > And I tried to add some comments for this new callback to distinguish it from > > ctx->update_progress. > > Comments don't help when using cscope or some such code browsing tool. > Better to use a different variable name. > + /* + * Callback to be called when updating progress during sending data of a + * transaction (and its subtransactions) to the output plugin. + */ + ReorderBufferUpdateProgressCB update_progress; Are you suggesting changing the name of the above variable? If so, how about apply_progress, progress, or updateprogress? If you don't like any of these then feel free to suggest something else. If we change the variable name then accordingly, we need to update ReorderBufferUpdateProgressCB as well. -- With Regards, Amit Kapila.
Commits
-
Fix the logical replication timeout during large DDLs.
- 8c58624df462 16.0 landed
-
Fix the logical replication timeout during large transactions.
- f95d53eded55 15.0 landed
- d6da71fa8f28 14.4 landed
- 55558df23741 13.8 landed
- f832b5007c1c 12.12 landed
- 87c1dd246af8 11.17 landed
- a4015ec0375d 10.22 landed
-
Rethink the delay-checkpoint-end mechanism in the back-branches.
- 10520f434687 14.3 cited
-
Revert "Logical decoding of sequences"
- 2c7ea57e56ca 15.0 cited
-
Skip empty transactions for logical replication.
- d5a9d86d8ffc 15.0 cited
-
Allow specifying column lists for logical replication
- 923def9a533a 15.0 cited
-
Add decoding of sequences to built-in replication
- 75b1521dae1f 15.0 cited
-
Fix ABI break introduced by commit 4daa140a2f.
- 56e366f6757d 13.4 cited
-
Lag tracking for logical replication
- 024711bb5446 10.0 cited