Horton

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

Andres Freund <andres@anarazel.de>

View thread
From: Andres Freund <andres@anarazel.de>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Kuntal Ghosh <kuntalghosh.2007@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Noah Misch <noah@leadboat.com>, Amit Khandekar <amitdkhan.pg@gmail.com>, Alvaro Herrera from 2ndQuadrant <alvherre@alvh.no-ip.org>, Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Robert Haas <robertmhaas@gmail.com>, Thomas Munro <thomas.munro@gmail.com>
Date: 2020-02-18T06:03:53Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. When a TAP file has non-zero exit status, retain temporary directories.

  2. Fix running out of file descriptors for spill files.

  3. Track statistics for spilling of changes from ReorderBuffer.

  4. Handle ReadFile() EOF correctly on Windows.

  5. Add logical_decoding_work_mem to limit ReorderBuffer memory usage.

  6. Generational memory allocator

  7. Support retaining data dirs on successful TAP tests

On 2020-02-18 11:20:17 +0530, Amit Kapila wrote:
> Andres, any objections on proceeding with Kuntal's patch for
> back-branches (10, 9.6 and 9.5)?

Yes. In my past experiments that lead to *terrible* allocator
performance due to fragmentation. Like, up to 90% of the time spent in
aset.c.  Try a workload with a number of overlapping transactions that
have different tuple sizes.

I'm not even sure it's the right thing to do anything in the back
branches to be honest. If somebody hits this badly they likely have done
so before, and they at least have the choice to upgrade, but if we
regress performance for more people...