Horton

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

Noah Misch <noah@leadboat.com>

View thread
From: Noah Misch <noah@leadboat.com>
To: Amit Kapila <amit.kapila16@gmail.com>, Amit Khandekar <amitdkhan.pg@gmail.com>
Cc: Alvaro Herrera from 2ndQuadrant <alvherre@alvh.no-ip.org>, Andres Freund <andres@anarazel.de>, Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Robert Haas <robertmhaas@gmail.com>, Thomas Munro <thomas.munro@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Tomas Vondra <tomas.vondra@2ndquadrant.com>
Date: 2020-01-04T18:51:48Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. When a TAP file has non-zero exit status, retain temporary directories.

  2. Fix running out of file descriptors for spill files.

  3. Track statistics for spilling of changes from ReorderBuffer.

  4. Handle ReadFile() EOF correctly on Windows.

  5. Add logical_decoding_work_mem to limit ReorderBuffer memory usage.

  6. Generational memory allocator

  7. Support retaining data dirs on successful TAP tests

On Fri, Jan 03, 2020 at 02:20:09PM +0530, Amit Khandekar wrote:
> On Fri, 3 Jan 2020 at 10:19, Amit Kapila <amit.kapila16@gmail.com> wrote:
> > On Fri, Jan 3, 2020 at 8:29 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >> I see one failure in REL_10_STABLE [1] which seems to be due to this commit:
> >
> > I tried this test on my CentOs and Power8 machine more than 50 times, but couldn't reproduce it.  So, adding Noah to see if he can try this test [1] on his machine (tern) and get stack track or some other information?
> >
> > [1] - make -C src/test/recovery/ check PROVE_TESTS=t/006_logical_decoding.pl
> 
> I also tested multiple times using PG 10 branch; also tried to inject
> an error so that PG_CATCH related code also gets covered, but
> unfortunately didn't get the crash on my machine. I guess, we will
> have to somehow get the stacktrace.

I have buildfarm member tern running this test in a loop.  In the 290
iterations so far, it hasn't failed.  I've leave it running for another week
or so.

The buildfarm client can capture stack traces, but it currently doesn't do so
for TAP test suites (search the client code for get_stack_trace).  If someone
feels like writing a fix for that, it would be a nice improvement.  Perhaps,
rather than having the client code know all the locations where core files
might appear, failed runs should walk the test directory tree for core files?