Horton

Re: hung backends stuck in spinlock heavy endless loop

Merlin Moncure <mmoncure@gmail.com>

View thread View on postgr.es
From: Merlin Moncure <mmoncure@gmail.com>
To: Andres Freund <andres@2ndquadrant.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2015-01-13T22:40:20Z
Lists: pgsql-hackers 
On Tue, Jan 13, 2015 at 4:33 PM, Andres Freund <andres@2ndquadrant.com> wrote:
> Hi,
>
> On 2015-01-13 16:29:51 -0600, Merlin Moncure wrote:
>> On my workstation today (running vanilla 9.4.0) I was testing some new
>> code that does aggressive parallel loading to a couple of tables. It
>> ran ok several dozen times and froze up with no external trigger.
>> There were at most 8 active backends that were stuck (the loader is
>> threaded to a cap) -- each query typically resolves in a few seconds
>> but they were hung for 30 minutes+.
>
> Interesting.
>
>> Had to do restart immediate as
>> backends were not responding to cancel...but I snapped a 'perf top'
>> before I did so.  The results were interesting so I'm posting them
>> here.  So far I have not been able to reproduce...FYI
>
> Can you compile postgres with -fno-omit-frame-pointer? Then, when this
> happens the next time, you can take a perf record -g, which will tell us
> which lock the contention is at.

will do, and I'll loop it for a while and see if I can get it to re-occur.

merlin

Commits

  1. Improve hash_create's API for selecting simple-binary-key hash functions.

  2. Fix WAL-logging of B-tree "unlink halfdead page" operation.

  3. Fix backup-block numbering in redo of b-tree split.

  4. Cleanup of new b-tree page deletion code.

  5. Fix bug in the new B-tree incomplete-split code.

  6. Fix bug in new B-tree page deletion code.

  7. Fix rm_desc routine of b-tree page delete records.

  8. Make the handling of interrupted B-tree page splits more robust.

  9. Fix race condition in B-tree page deletion.