Horton

Re: "could not reattach to shared memory" on buildfarm member dory

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Noah Misch <noah@leadboat.com>
Cc: Heath Lord <heath.lord@crunchydata.com>, Thomas Munro <thomas.munro@enterprisedb.com>, Stephen Frost <sfrost@snowman.net>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2018-05-01T03:33:32Z
Lists: pgsql-hackers 
Noah Misch <noah@leadboat.com> writes:
> On Mon, Apr 30, 2018 at 08:01:40PM -0400, Tom Lane wrote:
>> What seems like a plausible theory at this point is that the apparent
>> asynchronicity is due to the allocation being triggered by a different
>> thread, and the fact that our added monitoring code seems to make the
>> failure more likely can be explained by that code changing the timing.
>> But what thread could it be?  It doesn't really look to me like either
>> the signal thread or the timer thread could eat 4MB.  syslogger.c
>> also spawns a thread, on Windows, but AFAICS that's not being used in
>> this test configuration.

The 1-second wait doesn't seem to have changed things much, which puts
a hole in the idea that this is triggered by a thread spawned earlier
in backend process startup.

>> Maybe the reason dory is showing the problem
>> is something or other is spawning a thread we don't even know about?

> Likely some privileged daemon is creating a thread in every new process.

Yeah, I'm afraid that's the most likely theory at this point; it offers
an explanation why we're seeing this on dory and not other machines.
Although if the daemon were responding to process startup, wouldn't the
extra wait have given it time to do so?  There's still something that
doesn't add up here.

>> Any other ideas?

> PostgreSQL could retry the whole process creation, analogous to
> internal_forkexec() retries.

In the absence of any clearer theory about what's causing this,
that may be our only recourse.  Sure is ugly though.

			regards, tom lane

Commits

  1. Avoid "could not reattach" by providing space for concurrent allocation.

  2. Assert that pgwin32_signal_initialize() has been called early enough.

  3. Remove investigative code for can't-reattach-to-shared-memory errors.

  4. Does it help to wait before reattaching?

  5. Map and unmap the shared memory block before risking VirtualFree.

  6. Further effort at preventing memory map dump from affecting the results.

  7. Remove Windows module-list-dumping code.

  8. Dump full memory maps around failing Windows reattach code.

  9. Get still more info about Windows can't-reattach-to-shared-memory errors.

  10. Get more info about Windows can't-reattach-to-shared-memory errors.

  11. Try to get some info about Windows can't-reattach-to-shared-memory errors.