(unnamed)

text/html

Filename: (unnamed)
Type: text/html
Part: 0
Message: Re: BUG #17695: Failed Assert in logical replication snapbuild.
<html>
<head>
    <meta http-equiv='Content-Type' content='text/html; charset=UTF-8'>
</head>
<body>
<style>
    font{
        line-height: 1.6;
    }
    ul,ol{
        padding-left: 20px;
        list-style-position: inside;
    }
</style>
<div style = 'font-family:Helvetica,Helvetica,微软雅黑, 宋体; line-height:1.6;'>
    <div ></div><div>
    <div>Thanks for your advice! I update my test script, now it can 100% reproduce problem in my computer.&nbsp;<br></div><div><br></div><div>The new script and fix code are both in the patch. However, I met new problem after adding fix code. It fails into a new trap following:</div><div><br></div><div>TRAP: FailedAssertion("TransactionIdPrecedesOrEquals(safeXid, snap-&gt;xmin)", File: "snapbuild.c", Line: 593, PID: 1576)<br></div>postgres: master: walsender postgres [local] CREATE_REPLICATION_SLOT(ExceptionalCondition+0xb9)[0xb1ca28]<br>postgres: master: walsender postgres [local] CREATE_REPLICATION_SLOT(SnapBuildInitialSnapshot+0x1b3)[0x8f3c79]<br>postgres: master: walsender postgres [local] CREATE_REPLICATION_SLOT[0x9136b9]<br>postgres: master: walsender postgres [local] CREATE_REPLICATION_SLOT(exec_replication_command+0x398)[0x91456a]<br>postgres: master: walsender postgres [local] CREATE_REPLICATION_SLOT(PostgresMain+0x7be)[0x984e4a]<br>postgres: master: walsender postgres [local] CREATE_REPLICATION_SLOT[0x8c0d5b]<br>postgres: master: walsender postgres [local] CREATE_REPLICATION_SLOT[0x8c06cd]<br>postgres: master: walsender postgres [local] CREATE_REPLICATION_SLOT[0x8bc9de]<br>postgres: master: walsender postgres [local] CREATE_REPLICATION_SLOT(PostmasterMain+0x117a)[0x8bc2b5]<br>postgres: master: walsender postgres [local] CREATE_REPLICATION_SLOT[0x7bdb13]<div><br></div><div>
    </div>
    <div id="ntes-pcmac-signature" style="font-family:">
    
    <div style="font-size:14px; padding: 0;  margin:0;">

    </div>
</div>
    The problem is not very hard to reproduce, and I’m fixing it now.</div><div><br></div><div>Regards,</div><div><br></div><div>--</div><div><div><span style="color: rgb(49, 53, 59); font-size: 14px;">bowenshi</span></div></div><div><span style="color: rgb(120, 120, 122);"><br></span></div><div><span style="color: rgb(120, 120, 122);">---- Replied Message ----</span></div><div class="J-reply ntes-mailmaster-quote" style="padding-top: 1px; padding-bottom: 1px">
  <div style="
      margin-bottom: 1em;
      font-size: 12px;
      line-height: 1.25;
      color: #232324;
      padding: 0.5em 0.25em;
      border-radius: 0.67em;
      background-color: #f0f0f0;
    ">
    <table width="100%" cellpadding="0" cellspacing="9" border="0">
      <tbody><tr class="firstRow">
        <td valign="top" style="
            width: 4.25em;
            font-size: 12px;
            line-height: 1.25;
            color: #78787a;
          ">
         From 
        </td>
        <td valign="top" style="
            font-size: 12px;
            line-height: 1.25;
            color: #232324;
            word-break: break-all;
          ">
            <a class="mail-from" style="color: #3061f2; text-decoration: none" href="mailto:sawada.mshk@gmail.com">Masahiko Sawada&lt;sawada.mshk@gmail.com&gt;
            </a>
        </td>
      </tr>
      <tr>
        <td valign="top" style="
            width: 4.25em;
            font-size: 12px;
            line-height: 1.25;
            color: #78787a;
          ">
         Date 
        </td>
        <td class="mail-date" valign="top" style="
            font-size: 12px;
            line-height: 1.25;
            color: #232324;
            word-break: break-all;
          ">
        11/25/2022 13:58
        </td>
      </tr>
      <tr>
        <td valign="top" style="
            width: 4.25em;
            font-size: 12px;
            line-height: 1.25;
            color: #78787a;
          ">
         To 
        </td>
        <td valign="top" style="
            font-size: 12px;
            line-height: 1.25;
            color: #232324;
            word-break: break-all;
          ">
         
          <a class="mail-to" style="color: #3061f2; text-decoration: none" href="mailto:zxwsbg@qq.com">
            &lt;zxwsbg@qq.com&gt;
            ,<br>
          </a>
          <a class="mail-to" style="color: #3061f2; text-decoration: none" href="mailto:pgsql-bugs@lists.postgresql.org">
            &lt;pgsql-bugs@lists.postgresql.org&gt;
            
          </a>
        </td>
      </tr>
      <tr>
        <td valign="top" style="
            width: 4.25em;
            font-size: 12px;
            line-height: 1.25;
            color: #78787a;
          ">
         Subject 
        </td>
        <td class="mail-subject" valign="top" style="
            font-size: 12px;
            line-height: 1.25;
            color: #232324;
            word-break: break-all;
          ">
              Re: BUG #17695: Failed Assert in logical replication snapbuild.
        </td>
      </tr>
    </tbody></table>
  </div>
  <div id="ntes-pcmail-quote" style="margin: 0; padding: 0; font-size: 14px; font-family: ''Helvetica','Microsoft Yahei', '微软雅黑'';">On Thu, Nov 24, 2022 at 7:28 PM PG Bug reporting form<br>&lt;noreply@postgresql.org&gt; wrote:<br><blockquote class="mmbqc1"><br> The following bug has been logged on the website:<br><br> Bug reference: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;17695<br> Logged by: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;施博文<br> Email address: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;zxwsbg@qq.com<br> PostgreSQL version: 14.6<br> Operating system: &nbsp;&nbsp;centos<br> Description:<br><br> In PG14 or higher version, I notice that SnapBuildRestore don't set<br> builder-&gt;next_phase_at=InvalidTransactionId .<br><br> But in SnapBuildSerialize function, the assert check this condition.<br><br> Assert(builder-&gt;next_phase_at == InvalidTransactionId);<br><br> This would cause some problems, and I have repeat it with the perl test case<br> which I would update later, the problem is following:<br><br> TRAP: FailedAssertion("builder-&gt;next_phase_at == InvalidTransactionId",<br> File: "snapbuild.c", Line: 1604, PID: 29974)<br> postgres: master: walsender postgres [local]<br> START_REPLICATION(ExceptionalCondition+0xb9)[0xb1c9bd]<br> postgres: master: walsender postgres [local] START_REPLICATION[0x8f548d]<br> postgres: master: walsender postgres [local]<br> START_REPLICATION(SnapBuildProcessRunningXacts+0x55)[0x8f4c5c]<br> postgres: master: walsender postgres [local] START_REPLICATION[0x8dd8be]<br> postgres: master: walsender postgres [local]<br> START_REPLICATION(LogicalDecodingProcessRecord+0xd1)[0x8dd243]<br> postgres: master: walsender postgres [local] START_REPLICATION[0x915eb1]<br> postgres: master: walsender postgres [local] START_REPLICATION[0x91520c]<br> postgres: master: walsender postgres [local] START_REPLICATION[0x913bdd]<br> postgres: master: walsender postgres [local]<br> START_REPLICATION(exec_replication_command+0x42c)[0x914593]<br> postgres: master: walsender postgres [local]<br> START_REPLICATION(PostgresMain+0x7be)[0x984ddf]<br> postgres: master: walsender postgres [local] START_REPLICATION[0x8c0d41]<br> postgres: master: walsender postgres [local] START_REPLICATION[0x8c06b3]<br> postgres: master: walsender postgres [local] START_REPLICATION[0x8bc9c4]<br> postgres: master: walsender postgres [local]<br> START_REPLICATION(PostmasterMain+0x117a)[0x8bc29b]<br> postgres: master: walsender postgres [local] START_REPLICATION[0x7bdaf9]<br> /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f815dd92555]<br> postgres: master: walsender postgres [local] START_REPLICATION[0x488d09]<br><br></blockquote><br>Thank you for reporting the issue!<br><br>I could reproduce this issue with a small change and the following steps:<br><br>1. Add sleep after setting SNAPBUILD_BUILDING_SNAPSHOT.<br><br>@@ -1409,6 +1409,9 @@ SnapBuildFindSnapshot(SnapBuild *builder,<br>XLogRecPtr lsn, xl_running_xacts *runn<br>                           running-&gt;xcnt, running-&gt;nextXid)));<br><br>        SnapBuildWaitSnapshot(running, running-&gt;nextXid);<br>+<br>+ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;elog(LOG, "sleep 10s");<br>+ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;pg_usleep(10 * 1000000);<br>    }<br><br>2. Create and start database instance and create a table and the<br>replication slot 's1'.<br>create table test (i int);<br>select pg_logical_replication_slot('s1', 'test_decoding');<br><br>3. Create another replication slot "s2" while a transaction is running.<br><br>tx-1:<br>begin;<br>insert into test values (1);<br><br>    tx-2:<br>    select pg_logical_replication_slot('s2', 'test_decoding');<br><br>checkpoint;<br>commit;<br><br>Note that replication slot creation will wait for 10 sec after<br>commiting the tx-1, please go to the next step once the slot has been<br>created.<br><br>4. Start decoding on slot 's2'.<br>select pg_logical_slot_get_changes('s2', null, null);<br><br>The logical decoding will sleep. Please go to the next step after<br>seeing the log "sleep 10s".<br><br>5. While the logical decoding started at step 4 is sleeping, start<br>decoding on slot 's1'.<br>select pg_logical_slot_get_changes('s1', null, null);<br><br>6. After the decoding on slot 's1' wakes up, it fails due to the<br>assertion check.<br><br>This scenario simulates the case where the logical decoding (re)starts<br>where RUNNING_XACTS record having a running transactions and it<br>restores the serialized snapshot when decoding the next RUNNING_XACTS<br>record. Since we don't reset builder-&gt;next_phase_at when restoring a<br>serialized snapshot, the assertion check in SnapBuildSerialize fails.<br><br>Regarding the proposed patch, I've confirmed it fixes this issue. But<br>I think it's better to reset builder-&gt;next_phase_at right after the<br>following assertion check:<br><br>   /* consistent snapshots have no next phase */<br>   Assert(ondisk.builder.next_phase_at == InvalidTransactionId);<br><br>I could not reproduce this issue with your script in my environment. I<br>think it's better to include the reproducible test case in the patch<br>but I'm not sure how to do that without adding sleep/gdb attach.<br><br>Regards,<br><br>-- <br>Masahiko Sawada<br>Amazon Web Services: https://aws.amazon.com</div>
</div>
<!--�-->
</div>
</body>
</html>