Re: [HACKERS] logical decoding of two-phase transactions
Peter Smith <smithpb2250@gmail.com>
Attachments
- v16-0002-Support-2PC-txn-pgoutput.patch (application/octet-stream) patch v16-0002
- v16-0001-Support-2PC-txn-spoolfile.patch (application/octet-stream) patch v16-0001
- v16-0003-Support-2PC-txn-subscriber-tests.patch (application/octet-stream) patch v16-0003
Hi Amit
I have rebased, split, and addressed (most of) the review comments of
the v15-0003 patch.
So the previous v15-0003 patch is now split into three as follows:
- v16-0001-Support-2PC-txn-spoolfile.patch
- v16-0002-Support-2PC-txn-pgoutput.patch
- v16-0003-Support-2PC-txn-subscriber-tests.patch
PSA.
Of course the previous v15-0001 and v15-0002 are still required before
applying these v16 patches. Later (v17?) we will combine these again
with what Ajin is currently working on to give the full suite of
patches which will have a consistent version number.
On Tue, Nov 3, 2020 at 4:41 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> Few Comments on v15-0003-Support-2PC-txn-pgoutput
> ===============================================
> 1. This patch needs to be rebased after commit 644f0d7cc9 and requires
> some adjustments accordingly.
Done.
>
> 2.
> if (flags != 0)
> elog(ERROR, "unrecognized flags %u in commit message", flags);
>
> +
> /* read fields */
> commit_data->commit_lsn = pq_getmsgint64(in);
>
> Spurious line.
Fixed.
>
> 3.
> @@ -720,6 +722,7 @@ apply_handle_commit(StringInfo s)
> replorigin_session_origin_timestamp = commit_data.committime;
>
> CommitTransactionCommand();
> +
> pgstat_report_stat(false);
>
> Spurious line
Fixed.
>
> 4.
> +static void
> +apply_handle_prepare_txn(LogicalRepPrepareData * prepare_data)
> +{
> + Assert(prepare_data->prepare_lsn == remote_final_lsn);
> +
> + /* The synchronization worker runs in single transaction. */
> + if (IsTransactionState() && !am_tablesync_worker())
> + {
> + /* End the earlier transaction and start a new one */
> + BeginTransactionBlock();
> + CommitTransactionCommand();
> + StartTransactionCommand();
>
> There is no explanation as to why you want to end the previous
> transaction and start a new one. Even if we have to do so, we first
> need to call BeginTransactionBlock before CommitTransactionCommand.
TODO
>
> 5.
> - * Handle STREAM COMMIT message.
> + * Common spoolfile processing.
> + * Returns how many changes were applied.
> */
> -static void
> -apply_handle_stream_commit(StringInfo s)
> +static int
> +apply_spooled_messages(TransactionId xid, XLogRecPtr lsn)
> {
> - TransactionId xid;
>
> Can we have a separate patch for this as this can be committed before
> main patch. This is a refactoring required for the main patch.
Done.
>
> 6.
> @@ -57,7 +63,8 @@ static void pgoutput_stream_abort(struct
> LogicalDecodingContext *ctx,
> static void pgoutput_stream_commit(struct LogicalDecodingContext *ctx,
> ReorderBufferTXN *txn,
> XLogRecPtr commit_lsn);
> -
> +static void pgoutput_stream_prepare_txn(LogicalDecodingContext *ctx,
> + ReorderBufferTXN *txn, XLogRecPtr prepare_lsn);
>
> Spurious line removal.
Fixed.
---
Kind Regards,
Peter Smith.
Fujitsu Australia
Commits
-
Add prepare API support for streaming transactions in logical replication.
- 63cf61cdeb7b 15.0 landed
-
Doc: minor improvements for logical replication protocol documentation.
- a5cb4f9829fb 15.0 cited
-
Fix test failure in 021_twophase.pl.
- eaf5321c3524 15.0 landed
-
Refactor to make common functions in proto.c and worker.c.
- 91f9861242cd 15.0 landed
-
Unify PostgresNode's new() and get_new_node() methods
- 201a76183e20 15.0 cited
-
Fix potential buffer overruns in proto.c.
- 01c3adcdd85f 15.0 landed
-
Add support for prepared transactions to built-in logical replication.
- a8fd13cab0ba 15.0 cited
-
Refactor function parse_subscription_options.
- 8aafb0261675 15.0 cited
-
Allow enabling two-phase option via replication protocol.
- cda03cfed6b8 15.0 cited
-
Don't use Asserts to check for violations of replication protocol.
- fe6a20ce54cb 14.0 cited
-
Improve psql tab completion for options of subcriptions and publications
- d08237b5b494 14.0 cited
-
Rearrange logrep worker's snapshot handling some more.
- 3a09d75b4f6c 14.0 cited
-
doc: Update information of new messages for logical replication.
- 15c1a9d9cb76 14.0 cited
-
ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION
- 82ed7748b710 14.0 cited
-
Allow pgoutput to send logical decoding messages.
- ac4645c0157f 14.0 cited
-
Refactor function parse_output_parameters.
- 531737ddad21 14.0 cited
-
Avoid repeated decoding of prepared transactions after a restart.
- 8bdb1332eb51 14.0 cited
-
Fix an oversight in ReorderBufferFinishPrepared.
- ade89ba5f408 14.0 landed
-
Allow multiple xacts during table sync in logical replication.
- ce0fdbfe9722 14.0 cited
-
Fix replication of in-progress transactions in tablesync worker.
- 0926e96c4934 14.0 cited
-
Fix 'skip-empty-xacts' option in test_decoding for streaming mode.
- 9653f24ad830 14.0 cited
-
Use Enum for top level logical replication message types.
- 644f0d7cc9c2 14.0 cited
-
Add support for streaming to built-in logical replication.
- 464824323e57 14.0 cited
-
Fix the logical streaming test.
- 82a0ba7707e0 14.0 cited
-
Implement streaming mode in ReorderBuffer.
- 7259736a6e5b 14.0 cited
-
Extend the logical decoding output plugin API with stream methods.
- 45fdc9738b36 14.0 cited
-
Store 2PC GID in commit/abort WAL recs for logical decoding
- 1eb6d6527aae 11.0 cited