Re: Speedup twophase transactions

> On 13 Apr 2016, at 01:04, Michael Paquier <michael.paquier@gmail.com> wrote:
> 
> On Wed, Apr 13, 2016 at 1:53 AM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:
>>> On 12 Apr 2016, at 15:47, Michael Paquier <michael.paquier@gmail.com> wrote:
>>> 
>>> It looks to be the case... The PREPARE phase replayed after the
>>> standby is restarted in recovery creates a series of exclusive locks
>>> on the table created and those are not taken on HEAD. Once those are
>>> replayed the LOCK_STANDBY record is conflicting with it. In the case
>>> of the failure, the COMMIT PREPARED record cannot be fetched from
>>> master via the WAL stream so the relation never becomes visible.
>> 
>> Yep, it is. It is okay for prepared xact hold a locks for created/changed tables,
>> but code in standby_redo() was written in assumption that there are no prepared
>> xacts at the time of recovery. I’ll look closer at checkpointer code and will send
>> updated patch.
>> 
>> And thanks again.
> 
> That's too late for 9.6 unfortunately, don't forget to add that in the next CF!

Fixed patch attached. There already was infrastructure to skip currently
held locks during replay of standby_redo() and I’ve extended that with check for
prepared xids.

The reason why I’m still active on this thread is because I see real problems
in deploying 9.6 in current state. Let me stress my concern: current state of things
_WILL_BREAK_ async replication in case of substantial load of two phase
transactions on master. And a lot of J2EE apps falls into that category, as they
wrap most of their transactions in prepare/commit. Slave server just will always
increase it lag and will never catch up. It is possible to deal with that by switching
to synchronous replication or inserting triggers with pg_sleep on master, but it
doesn’t looks like normal behaviour of system.