Horton

Re: hung backends stuck in spinlock heavy endless loop

Peter Geoghegan <pg@heroku.com>

View thread View on postgr.es
From: Peter Geoghegan <pg@heroku.com>
To: Merlin Moncure <mmoncure@gmail.com>
Cc: Andres Freund <andres@2ndquadrant.com>, Tom Lane <tgl@sss.pgh.pa.us>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2015-01-14T20:24:08Z
Lists: pgsql-hackers 
On Wed, Jan 14, 2015 at 11:49 AM, Merlin Moncure <mmoncure@gmail.com> wrote:
> so it looks like nobody ever exits from _bt_moveright.   any last
> requests before I start bisecting down?

Could you write some code to print out the block number (i.e.
"BlockNumber blkno") if there are more than, say, 5 retries within
_bt_moveright()? You'll also want to print out the name of the index
(so rel->rd_rel->relname.data). That'll probably only trip when this
happens (5 retries are very unusual). Then, use contrib/pageinspect to
dump that block's contents (I think that'll work, since no one sits on
a buffer lock here).

I'd like to know what index we're dealing with here. I wonder if it's
"pg_attribute_relid_attnam_index".

So: call "bt_page_stats('pg_whatever', n)", n being the relevant block
number. Try and do the same with the right link (which bt_page_stats()
will show). Then try and call bt_page_items() against one or both of
those blocks/pages. I'd like to see what that shows. I'm wondering if
some invariant has been violated.

Which _bt_moveright() caller are we talking about here, BTW? I guess
it's just the _bt_first()/ _bt_search() one. That might vary, of
course.

-- 
Peter Geoghegan

Commits

  1. Improve hash_create's API for selecting simple-binary-key hash functions.

  2. Fix WAL-logging of B-tree "unlink halfdead page" operation.

  3. Fix backup-block numbering in redo of b-tree split.

  4. Cleanup of new b-tree page deletion code.

  5. Fix bug in the new B-tree incomplete-split code.

  6. Fix bug in new B-tree page deletion code.

  7. Fix rm_desc routine of b-tree page delete records.

  8. Make the handling of interrupted B-tree page splits more robust.

  9. Fix race condition in B-tree page deletion.