Re: Perform streaming logical transactions by background workers and parallel apply
Amit Kapila <amit.kapila16@gmail.com>
On Fri, Aug 26, 2022 at 9:30 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Thu, Aug 11, 2022 at 12:13 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > Since we will later consider applying non-streamed transactions in parallel, I
> > > think "apply streaming worker" might not be very suitable. I think PostgreSQL
> > > also has the worker "parallel worker", so for "apply parallel worker" and
> > > "apply background worker", I feel that "apply background worker" will make the
> > > relationship between workers more clear. ("[main] apply worker" and "apply
> > > background worker")
> > >
> >
> > But, on similar lines, we do have vacuumparallel.c for parallelizing
> > index vacuum. I agree with Kuroda-San on this point that the currently
> > proposed terminology doesn't sound to be very clear. The other options
> > that come to my mind are "apply streaming transaction worker", "apply
> > parallel worker" and file name could be applystreamworker.c,
> > applyparallel.c, applyparallelworker.c, etc. I see the point why you
> > are hesitant in calling it "apply parallel worker" but it is quite
> > possible that even for non-streamed xacts, we will share quite some
> > part of this code.
>
> I think the "apply streaming transaction worker" is a good option
> w.r.t. what we are currently doing but then in the future, if we want
> to apply normal transactions in parallel then we will have to again
> change the name. So I think "apply parallel worker" might look
> better and the file name could be "applyparallelworker.c" or just
> "parallelworker.c". Although "parallelworker.c" file name is a bit
> generic but we already have worker.c so w.r.t that "parallelworker.c"
> should just look fine.
>
Yeah based on that theory, we can go with parallelworker.c but my vote
is to go with applyparallelworker.c among the above as that is more
clear. I feel worker.c is already not a very good name where we are
doing the work related to apply, so it won't be advisable to go down
that path further.
--
With Regards,
Amit Kapila.
Commits
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Fix invalid memory access during the shutdown of the parallel apply worker.
- 3d144c6c8602 16.0 landed
-
Fix assertion failure in apply worker.
- de63f8dadee4 16.0 landed
-
Use elog to report unexpected action in handle_streamed_transaction().
- 781ac42d43ab 16.0 landed
-
Use appropriate wait event when sending data in the apply worker.
- d9d7fe68d35e 16.0 landed
-
Allow the logical_replication_mode to be used on the subscriber.
- 9f2213a7c575 16.0 landed
-
Rename GUC logical_decoding_mode to logical_replication_mode.
- 1e8b61735cfb 16.0 landed
-
Display the leader apply worker's PID for parallel apply workers.
- d540a02a724b 16.0 landed
-
Improve the code to decide and process the apply action.
- c981d9145dea 16.0 landed
-
Document the newly added wait events added by commit 216a784829.
- cd06ccd78fcf 16.0 landed
-
Perform apply of large transactions by parallel workers.
- 216a784829c2 16.0 landed
-
Wake up a subscription's replication worker processes after DDL.
- c6e1f62e2cee 16.0 cited
-
Add copyright notices to meson files
- 8284cf5f746f 16.0 cited
-
Better document logical replication parameters
- a8500750ca0a 16.0 cited
-
Add a common function to generate the origin name.
- 776e1c8a5d14 16.0 cited
-
Harmonize parameter names in storage and AM code.
- bfcf1b34805f 16.0 cited
-
Avoid using list_length() to test for empty list.
- efd0c16becbf 16.0 cited
-
Improve two comments related to a boolean DefElem's value
- 8445f5a21d40 16.0 cited
-
Fix partition table's REPLICA IDENTITY checking on the subscriber.
- 26b3455afaa9 15.0 cited
-
Fix data inconsistency between publisher and subscriber.
- b7658c24c7c1 15.0 cited
-
Fix cache look-up failures while applying changes in logical replication.
- 5a97b1325453 15.0 cited