Re: Properly handle OOM death?

> On Mar 13, 2023, at 12:16 PM, Peter J. Holzer <hjp-pgsql@hjp.at> wrote:
> 
> On 2023-03-13 09:55:50 -0800, Israel Brewster wrote:
>> On Mar 13, 2023, at 9:43 AM, Peter J. Holzer <hjp-pgsql@hjp.at> wrote:
> The syslog should contain a list of all tasks prior to the kill. For
> example, I just provoked an OOM kill on my laptop and the syslog
> contains (among lots of others) these lines:
> 
> Mar 13 21:00:36 trintignant kernel: [112024.084117] [   2721]   126  2721    54563     2042   163840      555          -900 postgres
> Mar 13 21:00:36 trintignant kernel: [112024.084123] [   2873]   126  2873    18211       85   114688      594             0 postgres
> Mar 13 21:00:36 trintignant kernel: [112024.084128] [   2941]   126  2941    54592     1231   147456      565             0 postgres
> Mar 13 21:00:36 trintignant kernel: [112024.084134] [   2942]   126  2942    54563      535   143360      550             0 postgres
> Mar 13 21:00:36 trintignant kernel: [112024.084139] [   2943]   126  2943    54563     1243   139264      548             0 postgres
> Mar 13 21:00:36 trintignant kernel: [112024.084145] [   2944]   126  2944    54798      561   147456      545             0 postgres
> Mar 13 21:00:36 trintignant kernel: [112024.084150] [   2945]   126  2945    54563      215   131072      551             0 postgres
> Mar 13 21:00:36 trintignant kernel: [112024.084156] [   2956]   126  2956    18718      506   122880      553             0 postgres
> Mar 13 21:00:36 trintignant kernel: [112024.084161] [   2957]   126  2957    54672      269   139264      546             0 postgres
> 
> That's less helpful than it could be since all the postgres processes
> are just listed as "postgres" without arguments. However, it is very
> likely that the first one is actually the postmaster, because it has the
> lowest pid (and the other pids follow closely) and it has an OOM score
> of -900 as set in the systemd service file.
> 
> So I could compare the PID of the killed process with this list (in my
> case the killed process wasn't one of them but a test program which just
> allocates lots of memory).

Oh, interesting. I had just greped for ‘Killed process’, so I didn’t see those preceding lines 😛 Looking at that, I see two things:
1) The entries in my syslog all refer to an R process, not a postgresql process at all
2) The ‘Killed process’ entry *does* actually have the process name in it - it’s just since the process name was “R”, I wasn’t making the connection 😄
 
> 
>        hp
> 
> -- 
>   _  | Peter J. Holzer    | Story must make more sense than reality.
> |_|_) |                    |
> | |   | hjp@hjp.at <mailto:hjp@hjp.at>         |    -- Charles Stross, "Creative writing
> __/   | http://www.hjp.at/ |       challenge!"