Re: Skipping logical replication transactions on subscriber side

Noah Misch <noah@leadboat.com>

From: Noah Misch <noah@leadboat.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Euler Taveira <euler@eulerto.com>, "osumi.takamichi@fujitsu.com" <osumi.takamichi@fujitsu.com>, Peter Eisentraut <peter.eisentraut@enterprisedb.com>, "David G. Johnston" <david.g.johnston@gmail.com>, "tanghy.fnst@fujitsu.com" <tanghy.fnst@fujitsu.com>, vignesh C <vignesh21@gmail.com>, Greg Nancarrow <gregn4422@gmail.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, Alexey Lesovsky <lesovsky@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2022-04-05T00:21:07Z
Lists: pgsql-hackers
On Mon, Apr 04, 2022 at 06:55:45PM +0900, Masahiko Sawada wrote:
> On Mon, Apr 4, 2022 at 3:26 PM Noah Misch <noah@leadboat.com> wrote:
> > On Mon, Apr 04, 2022 at 08:20:08AM +0530, Amit Kapila wrote:
> > > How about a comment like: "It has to be kept at 8-byte alignment
> > > boundary so as to be accessed directly via C struct as it uses
> > > TYPALIGN_DOUBLE for storage which has 4-byte alignment on platforms
> > > like AIX."? Can you please suggest a better comment if you don't like
> > > this one?
> >
> > I'd write it like this, though I'm not sure it's an improvement on your words:
> >
> >   When ALIGNOF_DOUBLE==4 (e.g. AIX), the C ABI may impose 8-byte alignment on
> >   some of the C types that correspond to TYPALIGN_DOUBLE SQL types.  To ensure
> >   catalog C struct layout matches catalog tuple layout, arrange for the tuple
> >   offset of each fixed-width, attalign='d' catalog column to be divisible by 8
> >   unconditionally.  Keep such columns before the first NameData column of the
> >   catalog, since packagers can override NAMEDATALEN to an odd number.
> 
> Thanks!
> 
> >
> > The best place for such a comment would be in one of
> > src/test/regress/sql/*sanity*.sql, next to a test written to detect new
> > violations.
> 
> Agreed.
> 
> IIUC in the new test, we would need a new SQL function to calculate
> the offset of catalog columns including padding, is that right? Or do
> you have an idea to do that by using existing functionality?

Something like this:

select
  attrelid::regclass,
  attname,
  array(select typname
        from pg_type t join pg_attribute pa on t.oid = pa.atttypid
        where pa.attrelid = a.attrelid and pa.attnum > 0 and pa.attnum < a.attnum order by pa.attnum) AS types_before,
  (select sum(attlen)
   from pg_type t join pg_attribute pa on t.oid = pa.atttypid
   where pa.attrelid = a.attrelid and pa.attnum > 0 and pa.attnum < a.attnum) AS len_before
from pg_attribute a
join pg_class c on c.oid = attrelid
where attalign = 'd' and relkind = 'r' and attnotnull and attlen <> -1
order by attrelid::regclass::text, attnum;
    attrelid     │   attname    │                types_before                 │ len_before
─────────────────┼──────────────┼─────────────────────────────────────────────┼────────────
 pg_sequence     │ seqstart     │ {oid,oid}                                   │          8
 pg_sequence     │ seqincrement │ {oid,oid,int8}                              │         16
 pg_sequence     │ seqmax       │ {oid,oid,int8,int8}                         │         24
 pg_sequence     │ seqmin       │ {oid,oid,int8,int8,int8}                    │         32
 pg_sequence     │ seqcache     │ {oid,oid,int8,int8,int8,int8}               │         40
 pg_subscription │ subskiplsn   │ {oid,oid,name,oid,bool,bool,bool,char,bool} │         81
(6 rows)

That doesn't count padding, but hazardous column changes will cause a diff in
the output.



Commits

  1. Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.

  2. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  3. Add ALTER SUBSCRIPTION ... SKIP.

  4. Optionally disable subscriptions on error.

  5. Update docs of logical replication for commit 8d74fc96db.

  6. Respect permissions within logical replication.

  7. Fix regression test failure caused by commit 8d74fc96db.

  8. Add a view to show the stats of subscription workers.

  9. Add logical change details to logical replication worker errcontext.

  10. Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.

  11. Fix typo in protocol.sgml.

  12. Remove unused argument in apply_handle_commit_internal().

  13. Fix replication of in-progress transactions in tablesync worker.

  14. Reorder pg_sequence columns to avoid alignment issue