Horton

Re: Skipping logical replication transactions on subscriber side

Noah Misch <noah@leadboat.com>

View thread View on postgr.es
From: Noah Misch <noah@leadboat.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Euler Taveira <euler@eulerto.com>, "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, "David G. Johnston" <david.g.johnston@gmail.com>, "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, vignesh C <vignesh21@gmail.com>, Greg Nancarrow <gregn4422@gmail.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2022-04-04T02:31:28Z
Lists: pgsql-hackers 
On Mon, Apr 04, 2022 at 10:28:30AM +0900, Masahiko Sawada wrote:
> On Sun, Apr 3, 2022 at 9:45 AM Noah Misch <noah@leadboat.com> wrote:
> > On Sat, Apr 02, 2022 at 08:44:45PM +0900, Masahiko Sawada wrote:
> > > On Sat, Apr 2, 2022 at 7:04 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > > On Sat, Apr 2, 2022 at 1:43 PM Noah Misch <noah@leadboat.com> wrote:
> > > > > Some options:
> > > > > - Move subskiplsn after subdbid, so it's always aligned anyway.  I've
> > > > >   confirmed that this lets the test pass, in 44s.

> --- a/src/include/catalog/pg_subscription.h
> +++ b/src/include/catalog/pg_subscription.h
> @@ -54,6 +54,17 @@ CATALOG(pg_subscription,6100,SubscriptionRelationId) BKI_SHARED_RELATION BKI_ROW
>  
>  	Oid			subdbid BKI_LOOKUP(pg_database);	/* Database the
>  													 * subscription is in. */
> +
> +	/*
> +	 * All changes finished at this LSN are skipped.
> +	 *
> +	 * Note that XLogRecPtr, pg_lsn in the catalog, is 8-byte alignment
> +	 * (TYPALIGN_DOUBLE) and it does not match the alignment on some platforms
> +	 * such as AIX.  Therefore subskiplsn needs to be placed here so it is
> +	 * always aligned.

I'm reading this comment as saying that TYPALIGN_DOUBLE is always 8 bytes, but
the problem arises precisely because TYPALIGN_DOUBLE==4 on AIX.

On most hosts, the C alignment of an XLogRecPtr is 8 bytes, and
TYPALIGN_DOUBLE==8.  On AIX, C alignment is still 8 bytes, but
TYPALIGN_DOUBLE==4.  The tuples on disk and in shared buffers use
TYPALIGN_DOUBLE to decide how much padding to insert, and that amount of
padding needs to match the C alignment padding.  Placing the field here
reduces the padding to zero, making that invariant hold trivially.

> +	 */
> +	XLogRecPtr	subskiplsn;
> +
>  	NameData	subname;		/* Name of the subscription */
>  
>  	Oid			subowner BKI_LOOKUP(pg_authid); /* Owner of the subscription */
> @@ -71,9 +82,6 @@ CATALOG(pg_subscription,6100,SubscriptionRelationId) BKI_SHARED_RELATION BKI_ROW
>  	bool		subdisableonerr;	/* True if a worker error should cause the
>  									 * subscription to be disabled */
>  
> -	XLogRecPtr	subskiplsn;		/* All changes finished at this LSN are
> -								 * skipped */

Some code sites list pg_subscription fields in field order.  Please update
them so they continue to list fields in field order.  CreateSubscription() is
one example.

Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue