Re: Skipping logical replication transactions on subscriber side
Noah Misch <noah@leadboat.com>
On Mon, Jun 13, 2022 at 10:25:24AM -0400, Robert Haas wrote:
> On Sun, Apr 17, 2022 at 11:22 PM Noah Misch <noah@leadboat.com> wrote:
> > > Yes, but it could be false positives in some cases. For instance, the
> > > column {oid, bool, XLogRecPtr} should be okay on ALIGNOF_DOUBLE == 4
> > > and 8 platforms but the new test fails.
> >
> > I'm happy with that, because the affected author should look for padding-free
> > layouts before settling on your example layout. If the padding-free layouts
> > are all unacceptable, the author should update the expected sanity_check.out
> > to show the one row where the test "fails".
> Perhaps instead we ought to legislate that NAMEDATALEN must
> be a multiple of 8, or some such thing.
>
> The other constraint, that typalign='d' fields must always fall on an
> 8 byte boundary, is probably less annoying in practice, but it's easy
> to imagine a future catalog running into trouble. Let's say we want to
> introduce a new catalog that has only an Oid column and a float8
> column. Perhaps with 0-3 bool or uint8 columns as well, or with any
> number of NameData columns as well. Well, the only way to satisfy this
> constraint is to put the float8 column first and the Oid column after
> it, which immediately makes it look different from every other catalog
> we have.
> AFAICS, we could do that by:
>
> 1. De-supporting platforms that have this problem, or
> 2. Introducing new typalign values, as Noah proposed back on April 2, or
> 3. Somehow forcing values that are sometimes 4-byte aligned and
> sometimes 8-byte aligned to be 8-byte alignment on all platforms
On Mon, Jun 20, 2022 at 10:04:06AM -0400, Robert Haas wrote:
> On Mon, Jun 20, 2022 at 9:52 AM Peter Eisentraut
> <peter.eisentraut@enterprisedb.com> wrote:
> > That means changing the system's ABI, so in the extreme case you then
> > need to compile everything else to match as well.
>
> I think we wouldn't want to do that in a minor release, but doing it
> in a new major release seems fine -- especially if only AIX is
> affected.
"Everything" isn't limited to PostgreSQL. The Perl ABI exposes large structs
to plperl; a field of type double could require the AIX user to rebuild Perl
with the same compiler option.
Overall, this could be a textbook example of choosing between:
- Mild harm (unaesthetic column order) to many people.
- Considerable harm (dump/reload instead of pg_upgrade) to a small, unknown,
possibly-zero quantity of people.
Here's how I rank the options, from most-preferred to least-preferred:
1. Put new eight-byte fields at the front of each catalog, when in doubt.
2. On systems where double alignment differs from int64 alignment, require
NAMEDATALEN%8==0. Upgrading to v16 would require dump/reload for AIX users
changing NAMEDATALEN to conform to the new restriction.
3. Introduce new typalign values. Upgrading to v16 would require dump/reload
for all AIX users.
4. De-support AIX.
5. From above, "Somehow forcing values that are sometimes 4-byte aligned and
sometimes 8-byte aligned to be 8-byte alignment on all platforms".
Upgrading to v16 would require dump/reload for all AIX users.
6. Require -qalign=natural on AIX. Upgrading to v16 would require dump/reload
and possible system library rebuilds for all AIX users.
I gather (1) isn't at the top of your ranking, or you wouldn't have written
in. What do you think of (2)?
Commits
-
Test ALIGNOF_DOUBLE==4 compatibility under ALIGNOF_DOUBLE==8.
- c1da0acbb06e 15.0 landed
-
Reorder subskiplsn in pg_subscription to avoid alignment issues.
- 79b716cfb7a1 15.0 landed
-
Add ALTER SUBSCRIPTION ... SKIP.
- 208c5d65bbd6 15.0 landed
-
Optionally disable subscriptions on error.
- 705e20f8550c 15.0 cited
-
Update docs of logical replication for commit 8d74fc96db.
- 85c61ba8920b 15.0 landed
-
Respect permissions within logical replication.
- a2ab9c06ea15 15.0 cited
-
Fix regression test failure caused by commit 8d74fc96db.
- 41e66fee0516 15.0 landed
-
Add a view to show the stats of subscription workers.
- 8d74fc96db5f 15.0 landed
-
Add logical change details to logical replication worker errcontext.
- abc0910e2e0a 15.0 landed
-
Rename LOGICAL_REP_MSG_STREAM_END to LOGICAL_REP_MSG_STREAM_STOP.
- 4cd7a1896871 15.0 landed
-
Fix typo in protocol.sgml.
- e1915646658d 14.0 landed
- 0ac1aee0d7d8 15.0 landed
-
Remove unused argument in apply_handle_commit_internal().
- f4b939f1a372 14.0 landed
- 16bd4becee32 15.0 landed
-
Fix replication of in-progress transactions in tablesync worker.
- 0926e96c4934 14.0 cited
-
Reorder pg_sequence columns to avoid alignment issue
- f3b421da5f4a 10.0 cited