Horton

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

Alvaro Herrera <alvherre@2ndquadrant.com>

View thread
From: Alvaro Herrera <alvherre@2ndquadrant.com>
To: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
Cc: Amit Khandekar <amitdkhan.pg@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>, Thomas Munro <thomas.munro@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2019-11-20T13:54:11Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. When a TAP file has non-zero exit status, retain temporary directories.

  2. Fix running out of file descriptors for spill files.

  3. Track statistics for spilling of changes from ReorderBuffer.

  4. Handle ReadFile() EOF correctly on Windows.

  5. Add logical_decoding_work_mem to limit ReorderBuffer memory usage.

  6. Generational memory allocator

  7. Support retaining data dirs on successful TAP tests

On 2019-Nov-20, Juan José Santamaría Flecha wrote:

> I was not able to reproduce the Permission denied error with current HEAD,
> until I opened another CMD inside the "pg_replslot/regression_slot" folder.
> This will be problematic, is the deletion of the folder actually needed?

Yes :-(  The code assumes that if the directory is there, then it's
valid.  Trying to remove that assumption is probably a more invasive
fix.

I think ReplicationSlotDropAcquired is too pessimistic (no recourse if
the rename fails) and too optimistic (this will almost never happen).
We could change it so that the rename is retried a few times, and avoid
the failure.  (Naturally, the rmtree should also be retried.)  The code
seems written with the POSIX semantics in mind, but it seems easy to
improve.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services