Horton

Re: [HACKERS] Weaker shmem interlock w/o postmaster.pid

Noah Misch <noah@leadboat.com>

View thread View on postgr.es
From: Noah Misch <noah@leadboat.com>
To: Daniel Gustafsson <daniel@yesql.se>
Cc: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>, "9erthalion6@gmail.com" <9erthalion6@gmail.com>, "sfrost@snowman.net" <sfrost@snowman.net>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2019-04-04T02:05:43Z
Lists: pgsql-hackers 
On Mon, Apr 01, 2019 at 08:19:56AM +0000, Daniel Gustafsson wrote:
> On Monday, April 1, 2019 12:42 AM, Noah Misch <noah@leadboat.com> wrote:
> > On Fri, Mar 29, 2019 at 09:53:51AM +0000, Daniel Gustafsson wrote:
> 
> > > This seems like a case where it would be useful to log a shmdt() error or do
> > > an Assert() around the success of the operation perhaps?
> >
> > I'll add the same elog(LOG) we have at other shmdt() sites. I can't think of
> > a site where we Assert() about the results of a system call. While shmdt()
> > might be a justified exception, elog(LOG) seems reasonable.
> 
> Agreed, seems reasonable.

Pushed, but that broke two buildfarm members:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=idiacanthus&dt=2019-04-04%2000%3A33%3A14
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=komodoensis&dt=2019-04-04%2000%3A33%3A13

I think the problem arose because these animals run on the same machine, and
their test execution was synchronized to the second.  Two copies of the new
test ran concurrently.  It doesn't tolerate that, owing to expectations about
which shared memory keys are in use.  My initial thought is to fix this by
having a third postmaster that runs throughout the test and represents
ownership of a given port.  If that postmaster gets something other than the
first shm key pertaining to its port, switch ports and try again.

I'll also include fixes for the warnings Andres reported on the
pgsql-committers thread.

Commits

  1. Probe only 127.0.0.1 when looking for ports on Unix.

  2. Don't write to stdin of a test process that could have already exited.

  3. Test both 0.0.0.0 and 127.0.0.x addresses to find a usable port.

  4. MSYS: Translate REGRESS_SHLIB to a Windows file name.

  5. MSYS: Skip src/test/recovery/t/017_shm.pl.

  6. When Perl "kill(9, ...)" fails, try "pg_ctl kill".

  7. Consistently test for in-use shared memory.

  8. Revert "Consistently test for in-use shared memory."

  9. Silence -Wimplicit-fallthrough in sysv_shmem.c.

  10. Make src/test/recovery/t/017_shm.pl safe for concurrent execution.

  11. Update HINT for pre-existing shared memory block.

  12. Add WL_EXIT_ON_PM_DEATH pseudo-event.

  13. The default values for shared_buffers and max_connections are now 1000

  14. XLOG (and related) changes:

  15. Significant cleanups in SysV IPC handling (shared mem and semaphores).