Horton

Re: [HACKERS] make async slave to wait for lsn to be replayed

Alexander Korotkov <aekorotkov@gmail.com>

View thread View on postgr.es
From: Alexander Korotkov <aekorotkov@gmail.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Kevin Hale Boyes <kcboyes@gmail.com>, Kartyshov Ivan <i.kartyshov@postgrespro.ru>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, hlinnaka@iki.fi, alvherre@alvh.no-ip.org, pashkin.elfe@gmail.com, bharath.rupireddyforpostgres@gmail.com, euler@eulerto.com, thomas.munro@gmail.com, peter@eisentraut.org, amit.kapila16@gmail.com, dilipbalaut@gmail.com, smithpb2250@gmail.com, pgsql-hackers@lists.postgresql.org
Date: 2024-08-06T10:23:35Z
Lists: pgsql-hackers 
On Tue, Aug 6, 2024 at 11:18 AM Alexander Korotkov <aekorotkov@gmail.com>
wrote:
> On Tue, Aug 6, 2024 at 8:36 AM Michael Paquier <michael@paquier.xyz>
wrote:
> > On Tue, Aug 06, 2024 at 05:17:10AM +0300, Alexander Korotkov wrote:
> > > The 0001 patch is intended to improve this situation.  Actually, it's
> > > not right to just put RecoveryInProgress() after
> > > GetXLogReplayRecPtr(), because more wal could be replayed between
> > > these calls.  Instead we need to recheck GetXLogReplayRecPtr() after
> > > getting negative result of RecoveryInProgress() because WAL replay
> > > position couldn't get updated after.
> > > 0002 patch comprises fix for the header comment of
WaitLSNSetLatches() function
> > > 0003 patch comprises tests for pg_wal_replay_wait() errors.
> >
> > Before adding more tests, could it be possible to stabilize what's in
> > the tree?  drongo has reported one failure with the recovery test
> > 043_wal_replay_wait.pl introduced recently by 3c5db1d6b016:
> >
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2024-08-05%2004%3A24%3A54
>
> Thank you for pointing!
> Surely, I'll fix this before.

Something breaks in these lines during second iteration of the loop.
"SELECT pg_current_wal_insert_lsn()" has been queried from primary, but
standby didn't receive "CALL pg_wal_replay_wait('...');"

for (my $i = 0; $i < 5; $i++)
{
    print($i);
    $node_primary->safe_psql('postgres',
        "INSERT INTO wait_test VALUES (${i});");
    my $lsn =
      $node_primary->safe_psql('postgres',
        "SELECT pg_current_wal_insert_lsn()");
    $psql_sessions[$i] = $node_standby1->background_psql('postgres');
    $psql_sessions[$i]->query_until(
        qr/start/, qq[
        \\echo start
        CALL pg_wal_replay_wait('${lsn}');
        SELECT log_count(${i});
    ]);
}

I wonder what could it be.  Probably something hangs inside launching
background psql...  I'll investigate this more.

------
Regards,
Alexander Korotkov
Supabase

Commits

  1. Ensure standby promotion point in 043_wal_replay_wait.pl

  2. Minor cleanup related to pg_wal_replay_wait() procedure

  3. Adjust pg_wal_replay_wait() procedure behavior on promoted standby

  4. pg_wal_replay_wait(): Fix typo in the doc

  5. Implement pg_wal_replay_wait() stored procedure

  6. Revert: Implement pg_wal_replay_wait() stored procedure

  7. Call WaitLSNCleanup() in AbortTransaction()

  8. Clarify what is protected by WaitLSNLock

  9. Use an LWLock instead of a spinlock in waitlsn.c

  10. Use the pairing heap instead of a flat array for LSN replay waiters

  11. Minor improvements for waitlsn.c

  12. Make the order of the header file includes consistent