Horton

Re: Hot Backup with rsync fails at pg_clog if under load

Aidan Van Dyk <aidan@highrise.ca>

View thread
From: Aidan Van Dyk <aidan@highrise.ca>
To: Simon Riggs <simon@2ndquadrant.com>
Cc: Florian Pflug <fgp@phlo.org>, Daniel Farina <daniel@heroku.com>, Chris Redekop <chris@replicon.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2011-10-26T11:54:16Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Derive oldestActiveXid at correct time for Hot Standby.

  2. Start Hot Standby faster when initial snapshot is incomplete.

  3. Fix timing of Startup CLOG and MultiXact during Hot Standby

On Wed, Oct 26, 2011 at 7:43 AM, Simon Riggs <simon@2ndquadrant.com> wrote:

>> It's very likely that it's a PostgreSQL problem, though. It's probably
>> not a pilot error since it happens even for backups taken with pg_basebackup(),
>> so the only explanation other than a PostgreSQL bug is broken hardware or
>> a pretty serious kernel/filesystem bug.
>
> The way forwards here is for someone to show the clog file that causes
> the error and find out why the call to read() fails.

Sorry, I thought the problem was obvious.  Either that, of I've
completely missed something in these threads...  I'll admit to not
following this one very closely anymore...

When the backup started, the clog was small.  So on the "recovering
instance", the clog is small.  PostgreSQL is supposed to be able to
deal with any file as it was when the backup starts.

When the backup is stopped, clog is big.  But that file was copied
after the backup was started, not after the backup finished.  So its
size is only guarenteed to be as big as it was when the backup
started.  Recovery is responsible for extending it as it was extended
during the backup period on the master.

The read fails because their is no data at the location it's trying to
read from, because clog hasn't been extended yet by recovery.

a.
-- 
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.