Horton

Re: hung backends stuck in spinlock heavy endless loop

Merlin Moncure <mmoncure@gmail.com>

View thread View on postgr.es
From: Merlin Moncure <mmoncure@gmail.com>
To: Andres Freund <andres@2ndquadrant.com>
Cc: Heikki Linnakangas <hlinnakangas@vmware.com>, Peter Geoghegan <pg@heroku.com>, Tom Lane <tgl@sss.pgh.pa.us>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2015-01-15T23:00:37Z
Lists: pgsql-hackers 
On Thu, Jan 15, 2015 at 4:03 PM, Merlin Moncure <mmoncure@gmail.com> wrote:
> On Thu, Jan 15, 2015 at 1:32 PM, Merlin Moncure <mmoncure@gmail.com> wrote:
>> Since it's possible the database is a loss, do you see any value in
>> bootstrappinng it again with checksums turned on?  One point of note
>> is that this is a brand spanking new SSD, maybe we nee to rule out
>> hardware based corruption?
>
> hm!  I bootstrapped a new database with checksums on, and lo,
>
> [cds2 18347 2015-01-15 15:58:29.955 CST 1779]WARNING:  page
> verification failed, calculated checksum 28520 but expected 28541
> [cds2 18347 2015-01-15 15:58:29.955 CST 1779]CONTEXT:  SQL statement
> "COPY lease_delete FROM
> '/tmp/C1188732_onesiteproperty.lease_delete.csv' CSV HEADER"

Running this test on another set of hardware to verify -- if this
turns out to be a false alarm which it may very well be, I can only
offer my apologies!  I've never had a new drive fail like that, in
that manner.  I'll burn the other hardware in overnight and report
back.

merlin

Commits

  1. Improve hash_create's API for selecting simple-binary-key hash functions.

  2. Fix WAL-logging of B-tree "unlink halfdead page" operation.

  3. Fix backup-block numbering in redo of b-tree split.

  4. Cleanup of new b-tree page deletion code.

  5. Fix bug in the new B-tree incomplete-split code.

  6. Fix bug in new B-tree page deletion code.

  7. Fix rm_desc routine of b-tree page delete records.

  8. Make the handling of interrupted B-tree page splits more robust.

  9. Fix race condition in B-tree page deletion.