Re: logical decoding / rewrite map vs. maxAllocatedDescs

Tomas Vondra <tomas.vondra@2ndquadrant.com> writes:
> While investigating an issue with rewrite maps in logical decoding, I 
> found it's pretty darn trivial to hit this:
>      ERROR:  53000: exceeded maxAllocatedDescs (492) while trying to open
>      file "pg_logical/mappings/map-4000-4eb-1_60DE1E08-5376b5-537c6b"

> This happens because we seem to open the mapping files and keep them 
> open until the next checkpoint, at which point we fsync + close.

It sounds like whoever wrote that code was completely impervious to the
API spec for AllocateFile():

 * Note that files that will be open for any significant length of time
 * should NOT be handled this way, since they cannot share kernel file
 * descriptors with other files; there is grave risk of running out of FDs
 * if anyone locks down too many FDs.

Perhaps these files should be accessed through fd.c's VFD infrastructure
rather than ignoring it.  Reinventing it is especially not the way to go.

> Furthermore, the situation is made worse by opening the same file 
> repeatedly, under different file descriptor.

That's just stupid :-(

Note that the limit on maxAllocatedDescs is based on the kernel limit
on how many files a process can have open, so assuming you can increase
it is not going to end well.

			regards, tom lane