Horton

Re: "could not reattach to shared memory" on buildfarm member dory

Noah Misch <noah@leadboat.com>

View thread View on postgr.es
From: Noah Misch <noah@leadboat.com>
To: Thomas Munro <thomas.munro@enterprisedb.com>
Cc: Stephen Frost <sfrost@snowman.net>, Tom Lane <tgl@sss.pgh.pa.us>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2018-04-24T07:06:46Z
Lists: pgsql-hackers 
On Tue, Apr 24, 2018 at 11:37:33AM +1200, Thomas Munro wrote:
> On Tue, Apr 24, 2018 at 11:18 AM, Stephen Frost <sfrost@snowman.net> wrote:
> > Greetings,
> >
> > * Tom Lane (tgl@sss.pgh.pa.us) wrote:
> >> So far, dory has failed three times with essentially identical symptoms:
> >>
> >> 2018-04-23 19:57:10.624 GMT [2240] FATAL:  could not reattach to shared memory (key=0000000000000190, addr=00000000018E0000): error code 487
> >> 2018-04-23 15:57:10.657 EDT [8836] ERROR:  lost connection to parallel worker
> >> 2018-04-23 15:57:10.657 EDT [8836] STATEMENT:  select count(*) from tenk1 group by twenty;
> >> 2018-04-23 15:57:10.660 EDT [3820] LOG:  background worker "parallel worker" (PID 2240) exited with exit code 1
> >>
> >> Now how can this be?  We've successfully reserved and released the address
> >> range we want to use, so it *should* be free at the instant we try to map.
> >
> > Yeah, that's definitely interesting.
> 
> I wondered if another thread with the right timing could map something
> between the VirtualFree() and MapViewOfFileEx() calls, but  we don't
> create the Windows signal handling thread until a bit later.  Could
> there be any any other threads active in the process?
> 
> Maybe try asking what's mapped there with VirtualQueryEx() on failure?

+1.  An implementation of that:
https://www.postgresql.org/message-id/20170403065106.GA2624300%40tornado.leadboat.com

Commits

  1. Avoid "could not reattach" by providing space for concurrent allocation.

  2. Assert that pgwin32_signal_initialize() has been called early enough.

  3. Remove investigative code for can't-reattach-to-shared-memory errors.

  4. Does it help to wait before reattaching?

  5. Map and unmap the shared memory block before risking VirtualFree.

  6. Further effort at preventing memory map dump from affecting the results.

  7. Remove Windows module-list-dumping code.

  8. Dump full memory maps around failing Windows reattach code.

  9. Get still more info about Windows can't-reattach-to-shared-memory errors.

  10. Get more info about Windows can't-reattach-to-shared-memory errors.

  11. Try to get some info about Windows can't-reattach-to-shared-memory errors.