Horton

Re: [HACKERS] logical decoding of two-phase transactions

Masahiko Sawada <sawada.mshk@gmail.com>

View thread View on postgr.es
From: Masahiko Sawada <sawada.mshk@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Ajin Cherian <itsajin@gmail.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2020-12-17T06:43:11Z
Lists: pgsql-hackers 
On Wed, Dec 16, 2020 at 6:22 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Wed, Dec 16, 2020 at 1:04 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> > Thank you for updating the patch. I have two questions:
> >
> > -----
> > @@ -239,6 +239,19 @@ CREATE SUBSCRIPTION <replaceable
> > class="parameter">subscription_name</replaceabl
> >           </para>
> >          </listitem>
> >         </varlistentry>
> > +       <varlistentry>
> > +        <term><literal>two_phase</literal> (<type>boolean</type>)</term>
> > +        <listitem>
> > +         <para>
> > +          Specifies whether two-phase commit is enabled for this subscription.
> > +          The default is <literal>false</literal>.
> > +          When two-phase commit is enabled then the decoded
> > transactions are sent
> > +          to the subscriber on the PREPARE TRANSACTION. When
> > two-phase commit is not
> > +          enabled then PREPARE TRANSACTION and COMMIT/ROLLBACK PREPARED are not
> > +          decoded on the publisher.
> > +         </para>
> > +        </listitem>
> > +       </varlistentry>
> >
> > The user will need to specify the 'two_phase’ option on CREATE
> > SUBSCRIPTION. It would mean the user will need to control what data is
> > streamed both on publication side for INSERT/UPDATE/DELETE/TRUNCATE
> > and on subscriber side for PREPARE. Looking at the implementation of
> > the ’two_phase’ option of CREATE SUBSCRIPTION, it ultimately just
> > passes the ‘two_phase' option to the publisher. Why don’t we set it on
> > the publisher side?
> >
>
> There could be multiple subscriptions for the same publication, some
> want to decode the transaction at prepare time and others might want
> to decode at commit time only.  Also, one subscription could subscribe
> to multiple publications, so not sure if it is even feasible to set at
> publication level (consider one txn has changes belonging to multiple
> publications). This option controls how the data is streamed from a
> publication similar to other options like 'streaming'. Why do you
> think this should be any different?

Oh, I was thinking that the option controls what data is streamed
similar to the 'publish' option. But I agreed with you. As you
mentioned, it might be a problem if a subscription subscribes multiple
publications setting different ’two_phase’ options. Also in terms of
changing this option while streaming changes, it’s better to control
it on the subscriber side.

Regards,

-- 
Masahiko Sawada
EnterpriseDB:  https://www.enterprisedb.com/

Commits

  1. Add prepare API support for streaming transactions in logical replication.

  2. Doc: minor improvements for logical replication protocol documentation.

  3. Fix test failure in 021_twophase.pl.

  4. Refactor to make common functions in proto.c and worker.c.

  5. Unify PostgresNode's new() and get_new_node() methods

  6. Fix potential buffer overruns in proto.c.

  7. Add support for prepared transactions to built-in logical replication.

  8. Refactor function parse_subscription_options.

  9. Allow enabling two-phase option via replication protocol.

  10. Don't use Asserts to check for violations of replication protocol.

  11. Improve psql tab completion for options of subcriptions and publications

  12. Rearrange logrep worker's snapshot handling some more.

  13. doc: Update information of new messages for logical replication.

  14. ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION

  15. Allow pgoutput to send logical decoding messages.

  16. Refactor function parse_output_parameters.

  17. Avoid repeated decoding of prepared transactions after a restart.

  18. Fix an oversight in ReorderBufferFinishPrepared.

  19. Allow multiple xacts during table sync in logical replication.

  20. Fix replication of in-progress transactions in tablesync worker.

  21. Fix 'skip-empty-xacts' option in test_decoding for streaming mode.

  22. Use Enum for top level logical replication message types.

  23. Add support for streaming to built-in logical replication.

  24. Fix the logical streaming test.

  25. Implement streaming mode in ReorderBuffer.

  26. Extend the logical decoding output plugin API with stream methods.

  27. Store 2PC GID in commit/abort WAL recs for logical decoding