Re: [HACKERS] logical decoding of two-phase transactions

Peter Smith <smithpb2250@gmail.com>

From: Peter Smith <smithpb2250@gmail.com>

To: PostgreSQL Hackers <pgsql-hackers@postgresql.org>

Date: 2021-03-06T01:49:04Z

Lists: pgsql-hackers

Attachments

v50-0001-Refactor-spool-file-logic-in-worker.c.patch (application/octet-stream) patch v50-0001
v50-0002-Track-replication-origin-progress-for-rollbacks.patch (application/octet-stream) patch v50-0002
v50-0004-Add-two_phase-option-to-CREATE-REPLICATION-SLOT.patch (application/octet-stream) patch v50-0004
v50-0003-Add-support-for-apply-at-prepare-time-to-built-i.patch (application/octet-stream) patch v50-0003
v50-0005-Support-2PC-txn-subscriber-tests.patch (application/octet-stream) patch v50-0005
v50-0006-Support-2PC-txn-Subscription-option.patch (application/octet-stream) patch v50-0006
v50-0007-Tablesync-early-exit.patch (application/octet-stream) patch v50-0007
v50-0008-Fix-apply-worker-empty-prepare.patch (application/octet-stream) patch v50-0008
v50-0009-Fix-apply-worker-empty-prepare-dev-logs.patch (application/octet-stream) patch v50-0009

Please find attached the latest patch set v50*

Differences from v49* are:

* Rebased to HEAD @ today

* Patch 0008 "empty prepare" is updated to address the following
feedback comments:

From Amit @ 2021-03-03 [ak]
- (18) Fixed. Removed special cases in
prepare_spoolfile_replay_messages. Just dispatch all messages.
- (19) Fixed. Before replay the psf remote_final_lsn needs to be set
as commit_prepared's commit_lsn

From Vignesh @ 2021-03-05 [vc]
- (21) Fixed. Reworded comment to not refer to the fix as a "workaround".
- (25) Fixed. A comment line exceeds 80 chars.

-----
[ak] https://www.postgresql.org/message-id/CAA4eK1KhfzCYDmv17beC6wOX_5pL-MBNYBpMiLgxrdgF1yBYng%40mail.gmail.com
[vc] https://www.postgresql.org/message-id/CALDaNm1rRG2EUus%2BmFrqRzEshZwJZtxja0rn_n3qXGAygODfOA%40mail.gmail.com

Kind Regards,
Peter Smith.
Fujitsu Australia

Commits

Add prepare API support for streaming transactions in logical replication.
- 63cf61cdeb7b 15.0 landed
Doc: minor improvements for logical replication protocol documentation.
- a5cb4f9829fb 15.0 cited
Fix test failure in 021_twophase.pl.
- eaf5321c3524 15.0 landed
Refactor to make common functions in proto.c and worker.c.
- 91f9861242cd 15.0 landed
Unify PostgresNode's new() and get_new_node() methods
- 201a76183e20 15.0 cited
Fix potential buffer overruns in proto.c.
- 01c3adcdd85f 15.0 landed
Add support for prepared transactions to built-in logical replication.
- a8fd13cab0ba 15.0 cited
Refactor function parse_subscription_options.
- 8aafb0261675 15.0 cited
Allow enabling two-phase option via replication protocol.
- cda03cfed6b8 15.0 cited
Don't use Asserts to check for violations of replication protocol.
- fe6a20ce54cb 14.0 cited
Improve psql tab completion for options of subcriptions and publications
- d08237b5b494 14.0 cited
Rearrange logrep worker's snapshot handling some more.
- 3a09d75b4f6c 14.0 cited
doc: Update information of new messages for logical replication.
- 15c1a9d9cb76 14.0 cited
ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION
- 82ed7748b710 14.0 cited
Allow pgoutput to send logical decoding messages.
- ac4645c0157f 14.0 cited
Refactor function parse_output_parameters.
- 531737ddad21 14.0 cited
Avoid repeated decoding of prepared transactions after a restart.
- 8bdb1332eb51 14.0 cited
Fix an oversight in ReorderBufferFinishPrepared.
- ade89ba5f408 14.0 landed
Allow multiple xacts during table sync in logical replication.
- ce0fdbfe9722 14.0 cited
Fix replication of in-progress transactions in tablesync worker.
- 0926e96c4934 14.0 cited
Fix 'skip-empty-xacts' option in test_decoding for streaming mode.
- 9653f24ad830 14.0 cited
Use Enum for top level logical replication message types.
- 644f0d7cc9c2 14.0 cited
Add support for streaming to built-in logical replication.
- 464824323e57 14.0 cited
Fix the logical streaming test.
- 82a0ba7707e0 14.0 cited
Implement streaming mode in ReorderBuffer.
- 7259736a6e5b 14.0 cited
Extend the logical decoding output plugin API with stream methods.
- 45fdc9738b36 14.0 cited
Store 2PC GID in commit/abort WAL recs for logical decoding
- 1eb6d6527aae 11.0 cited