Horton

Thread

Commits

  1. Fix ReorderBuffer memory overflow check.

  1. FailedAssertion at ReorderBufferCheckMemoryLimit()

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2020-06-09T08:26:24Z

    Hi,

I encountered the following assertion failure when I changed
logical_decoding_work_mem to lower value while logical replication
is running. This happend in the master branch.

TRAP: FailedAssertion("rb->size < logical_decoding_work_mem * 1024L", File: "reorderbuffer.c", Line: 2403)
0   postgres                            0x000000010755bf80 ExceptionalCondition + 160
1   postgres                            0x00000001072d9f81 ReorderBufferCheckMemoryLimit + 257
2   postgres                            0x00000001072d9b74 ReorderBufferQueueChange + 228
3   postgres                            0x00000001072cd107 DecodeInsert + 391
4   postgres                            0x00000001072cc4ef DecodeHeapOp + 335
5   postgres                            0x00000001072cb9e4 LogicalDecodingProcessRecord + 196
6   postgres                            0x000000010730bf06 XLogSendLogical + 166
7   postgres                            0x000000010730b409 WalSndLoop + 217
8   postgres                            0x00000001073075fc StartLogicalReplication + 716
9   postgres                            0x0000000107305d38 exec_replication_command + 1192
10  postgres                            0x000000010737ea7f PostgresMain + 2463
11  postgres                            0x00000001072a9c4a BackendRun + 570
12  postgres                            0x00000001072a907b BackendStartup + 475
13  postgres                            0x00000001072a7fe1 ServerLoop + 593
14  postgres                            0x00000001072a5a5a PostmasterMain + 5898
15  postgres                            0x0000000107187b59 main + 761
16  libdyld.dylib                       0x00007fff6c00c3d5 start + 1


ReorderBufferCheckMemoryLimit() explains that it relies on
the following (commented) assumption. But this seems incorrect
when logical_decoding_work_mem is decreased. I wonder if we may
need to keep evicting the transactions until we don't exceed
memory limit.

	/*
	 * And furthermore, evicting the transaction should get us below the
	 * memory limit again - it is not possible that we're still exceeding the
	 * memory limit after evicting the transaction.
	 *
	 * This follows from the simple fact that the selected transaction is at
	 * least as large as the most recent change (which caused us to go over
	 * the memory limit). So by evicting it we're definitely back below the
	 * memory limit.
	 */
	Assert(rb->size < logical_decoding_work_mem * 1024L);

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

  2. Re: FailedAssertion at ReorderBufferCheckMemoryLimit()

    Amit Kapila <amit.kapila16@gmail.com> — 2020-06-09T08:58:19Z

    On Tue, Jun 9, 2020 at 1:56 PM Fujii Masao <masao.fujii@oss.nttdata.com> wrote:
>
> Hi,
>
> I encountered the following assertion failure when I changed
> logical_decoding_work_mem to lower value while logical replication
> is running. This happend in the master branch.
>
> TRAP: FailedAssertion("rb->size < logical_decoding_work_mem * 1024L", File: "reorderbuffer.c", Line: 2403)
..
>
>
> ReorderBufferCheckMemoryLimit() explains that it relies on
> the following (commented) assumption. But this seems incorrect
> when logical_decoding_work_mem is decreased.
>

Yeah, that could be a problem.

> I wonder if we may
> need to keep evicting the transactions until we don't exceed
> memory limit.
>

Yes, that should be the right fix here.

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

  3. Re: FailedAssertion at ReorderBufferCheckMemoryLimit()

    Amit Kapila <amit.kapila16@gmail.com> — 2020-06-10T03:00:15Z

    On Tue, Jun 9, 2020 at 2:28 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Tue, Jun 9, 2020 at 1:56 PM Fujii Masao <masao.fujii@oss.nttdata.com> wrote:
> >
>
> > I wonder if we may
> > need to keep evicting the transactions until we don't exceed
> > memory limit.
> >
>
> Yes, that should be the right fix here.
>

Can you please check whether the attached patch fixes the problem for you?

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

  4. Re: FailedAssertion at ReorderBufferCheckMemoryLimit()

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2020-06-10T03:45:40Z

    
On 2020/06/10 12:00, Amit Kapila wrote:
> On Tue, Jun 9, 2020 at 2:28 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>>
>> On Tue, Jun 9, 2020 at 1:56 PM Fujii Masao <masao.fujii@oss.nttdata.com> wrote:
>>>
>>
>>> I wonder if we may
>>> need to keep evicting the transactions until we don't exceed
>>> memory limit.
>>>
>>
>> Yes, that should be the right fix here.
>>
> 
> Can you please check whether the attached patch fixes the problem for you?

Thanks for the patch! The patch looks good to me.

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

  5. Re: FailedAssertion at ReorderBufferCheckMemoryLimit()

    Amit Kapila <amit.kapila16@gmail.com> — 2020-06-10T11:18:45Z

    On Wed, Jun 10, 2020 at 9:15 AM Fujii Masao <masao.fujii@oss.nttdata.com> wrote:
>
> On 2020/06/10 12:00, Amit Kapila wrote:
> > On Tue, Jun 9, 2020 at 2:28 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >>
> >> On Tue, Jun 9, 2020 at 1:56 PM Fujii Masao <masao.fujii@oss.nttdata.com> wrote:
> >>>
> >>
> >>> I wonder if we may
> >>> need to keep evicting the transactions until we don't exceed
> >>> memory limit.
> >>>
> >>
> >> Yes, that should be the right fix here.
> >>
> >
> > Can you please check whether the attached patch fixes the problem for you?
>
> Thanks for the patch! The patch looks good to me.
>

Thanks, pushed!

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com