Horton

Re: [HACKERS] logical decoding of two-phase transactions

Amit Kapila <amit.kapila16@gmail.com>

View thread View on postgr.es
From: Amit Kapila <amit.kapila16@gmail.com>
To: Ajin Cherian <itsajin@gmail.com>
Cc: Peter Smith <smithpb2250@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2020-12-15T12:43:48Z
Lists: pgsql-hackers 
On Mon, Dec 14, 2020 at 2:59 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>

Today, I looked at one of the issues discussed earlier in this thread
[1] which is that decoding can block (or deadlock can happen) when the
user explicitly locks the catalog relation (like Lock pg_class) or
perform Cluster on non-relmapped catalog relations (like Cluster
pg_trigger using pg_class_oid_index; and the user_table on which we
have performed any operation has a trigger) in the prepared xact. As
discussed previously, we don't have a problem when user tables are
exclusively locked because during decoding we don't acquire any lock
on those and in fact, we have a test case for the same in the patch.

In the previous discussion, most people seem to be of opinion that we
should document it in a category "don't do that", or prohibit to
prepare transactions that lock system tables in the exclusive mode as
any way that can block the entire system. The other possibility could
be that the plugin can allow enabling lock_timeout when it wants to
allow decoding of two-phase xacts and if the timeout occurs it tries
to fetch by disabling two-phase option provided by the patch.

I think it is better to document this as there is no realistic
scenario where it can happen. I also think separately (not as part of
this patch) we can investigate whether it is a good idea to prohibit
prepare for transactions that acquire exclusive locks on catalog
relations.

Thoughts?

[1] - https://www.postgresql.org/message-id/CAMGcDxf83P5SGnGH52%3D_0wRP9pO6uRWCMRwAA0nxKtZvir2_vQ%40mail.gmail.com

-- 
With Regards,
Amit Kapila.

Commits

  1. Add prepare API support for streaming transactions in logical replication.

  2. Doc: minor improvements for logical replication protocol documentation.

  3. Fix test failure in 021_twophase.pl.

  4. Refactor to make common functions in proto.c and worker.c.

  5. Unify PostgresNode's new() and get_new_node() methods

  6. Fix potential buffer overruns in proto.c.

  7. Add support for prepared transactions to built-in logical replication.

  8. Refactor function parse_subscription_options.

  9. Allow enabling two-phase option via replication protocol.

  10. Don't use Asserts to check for violations of replication protocol.

  11. Improve psql tab completion for options of subcriptions and publications

  12. Rearrange logrep worker's snapshot handling some more.

  13. doc: Update information of new messages for logical replication.

  14. ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION

  15. Allow pgoutput to send logical decoding messages.

  16. Refactor function parse_output_parameters.

  17. Avoid repeated decoding of prepared transactions after a restart.

  18. Fix an oversight in ReorderBufferFinishPrepared.

  19. Allow multiple xacts during table sync in logical replication.

  20. Fix replication of in-progress transactions in tablesync worker.

  21. Fix 'skip-empty-xacts' option in test_decoding for streaming mode.

  22. Use Enum for top level logical replication message types.

  23. Add support for streaming to built-in logical replication.

  24. Fix the logical streaming test.

  25. Implement streaming mode in ReorderBuffer.

  26. Extend the logical decoding output plugin API with stream methods.

  27. Store 2PC GID in commit/abort WAL recs for logical decoding