Horton

Re: Logical Replication of sequences

Peter Smith <smithpb2250@gmail.com>

View thread
From: Peter Smith <smithpb2250@gmail.com>
To: vignesh C <vignesh21@gmail.com>
Cc: Shlok Kyal <shlok.kyal.oss@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, Peter Eisentraut <peter@eisentraut.org>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Tomas Vondra <tomas.vondra@enterprisedb.com>, Euler Taveira <euler@eulerto.com>, Michael Paquier <michael@paquier.xyz>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>, "Jonathan S. Katz" <jkatz@postgresql.org>
Date: 2024-07-05T04:15:36Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Doc: Add documentation for sequence synchronization.

  2. Remove unused assignment in CREATE PUBLICATION grammar.

  3. Add seq_sync_error_count to subscription statistics.

  4. Fix few issues in commit 5509055d69.

  5. Add sequence synchronization for logical replication.

  6. Add worker type argument to logical replication worker functions.

  7. Introduce "REFRESH SEQUENCES" for subscriptions.

  8. Refactor logical worker synchronization code into a separate file.

  9. Standardize use of REFRESH PUBLICATION in code and messages.

  10. Add "ALL SEQUENCES" support to publications.

  11. Expose sequence page LSN via pg_get_sequence_data.

  12. Resume conflict-relevant data retention automatically.

  13. Post-commit review fixes for 228c370868.

  14. Generate GUC tables from .dat file

Hi Vignesh.

After applying the v20240703-0003 patch, I was always getting errors
when running the subscription TAP tests.

# +++ tap check in src/test/subscription +++
t/001_rep_changes.pl ............... ok
t/002_types.pl ..................... ok
t/003_constraints.pl ............... ok
t/004_sync.pl ...................... ok
t/005_encoding.pl .................. ok
t/006_rewrite.pl ................... ok
t/007_ddl.pl ....................... 3/?
#   Failed test 'Alter subscription set publication throws warning for
non-existent publication'
#   at t/007_ddl.pl line 67.
Bailout called.  Further testing stopped:  pg_ctl stop failed
# Tests were run but no plan was declared and done_testing() was not seen.
FAILED--Further testing stopped: pg_ctl stop failed
make: *** [check] Error 255

~~~

The publisher log shows an Assert TRAP occurred:

2024-07-04 18:15:40.089 AEST [745] mysub1 LOG:  statement: SELECT
DISTINCT s.schemaname, s.sequencename
  FROM pg_catalog.pg_publication_sequences s
WHERE s.pubname IN ('mypub', 'non_existent_pub', 'non_existent_pub1',
'non_existent_pub2')
TRAP: failed Assert("IsA(list, OidList)"), File:
"../../../src/include/nodes/pg_list.h", Line: 323, PID: 745

~~~

A debugging backtrace looks like below:

Core was generated by `postgres: publisher: walsender postgres
postgres [local] SELECT               '.
Program terminated with signal 6, Aborted.
#0  0x00007f36f44f02c7 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install
glibc-2.17-260.el7_6.6.x86_64 pcre-8.32-17.el7.x86_64
(gdb) bt
#0  0x00007f36f44f02c7 in raise () from /lib64/libc.so.6
#1  0x00007f36f44f19b8 in abort () from /lib64/libc.so.6
#2  0x0000000000bb8be1 in ExceptionalCondition (conditionName=0xc7aa6c
"IsA(list, OidList)",
    fileName=0xc7aa10 "../../../src/include/nodes/pg_list.h",
lineNumber=323) at assert.c:66
#3  0x00000000005f2c57 in list_nth_oid (list=0x27948f0, n=0) at
../../../src/include/nodes/pg_list.h:323
#4  0x00000000005f5491 in pg_get_publication_sequences
(fcinfo=0x2796a00) at pg_publication.c:1334
#5  0x0000000000763d10 in ExecMakeTableFunctionResult
(setexpr=0x27b2fd8, econtext=0x27b2ef8, argContext=0x2796900,
...

Something goes wrong indexing into that 'sequences' list.

1329 funcctx = SRF_PERCALL_SETUP();
1330 sequences = (List *) funcctx->user_fctx;
1331
1332 if (funcctx->call_cntr < list_length(sequences))
1333 {
1334 Oid relid = list_nth_oid(sequences, funcctx->call_cntr);
1335
1336 SRF_RETURN_NEXT(funcctx, ObjectIdGetDatum(relid));
1337 }

======

Perhaps now it is time to create a CF entry for this thread because
the cfbot could have detected the error earlier.

======
Kind Regards,
Peter Smith.
Fujitsu Australia