Horton

Re: stress test for parallel workers

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Justin Pryzby <pryzby@telsasoft.com>
Cc: pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@enterprisedb.com>
Date: 2019-07-23T17:28:47Z
Lists: pgsql-hackers 
Justin Pryzby <pryzby@telsasoft.com> writes:
> Does anyone have a stress test for parallel workers ?
> On a customer's new VM, I got this several times while (trying to) migrate their DB:

> < 2019-07-23 10:33:51.552 CDT postgres >FATAL:  postmaster exited during a parallel transaction

We've been seeing this irregularly in the buildfarm, too.  I've been
suspicious that it's from an OOM kill on the postmaster in the
buildfarm cases, but ...

> There's nothing in dmesg nor in postgres logs.

... you'd think an OOM kill would show up in the kernel log.
(Not necessarily in dmesg, though.  Did you try syslog?)

> Ideally a minimal test, since I'm apparently going to
> have to run under gdb to see how it's dying, or even what process is failing.

Like it told you, it's the postmaster that's going away.
That's Not Supposed To Happen, of course, but unfortunately Linux'
OOM kill heuristic preferentially targets the postmaster when
its children are consuming too many resources.

If that is the problem, there's some info on working around it at

https://www.postgresql.org/docs/current/kernel-resources.html#LINUX-MEMORY-OVERCOMMIT

			regards, tom lane

Commits

  1. In the postmaster, rely on the signal infrastructure to block signals.

  2. Paper over regression failures in infinite_recurse() on PPC64 Linux.

  3. Hack pg_ctl to report postmaster's exit status.

  4. Re-order some regression test scripts for more parallelism.